Hi, I'm Gauri Jaiswal.
|

FastAPI app exposes a POST /chat endpoint. On startup it pulls a model (gemma3:1b) into a local Ollama runtime, then forwards each request's message to the local model and returns the response.

How self-hosted inference differs from calling OpenAI/Gemini — no per-token cost, but you own the runtime and model management.

A plan → action → observe → output loop. The model is constrained to strict JSON output at each step; when it emits an 'action' step, the corresponding Python function actually runs and its result is fed back in as an 'observe' step before the loop continues.

Getting an LLM to reliably emit parseable JSON across multiple turns is harder than it looks — most of the iteration (visible in the repo's history) went into tightening the prompt rules, not the tool logic itself.

A system prompt requires the model to move through analyse → think → output → validate → result as separate JSON-formatted turns, one at a time, with the conversation history replayed on every call.

How much prompt structure actually changes model behavior — the step-by-step schema visibly slows the model down into more deliberate answers on multi-step problems.

PyPDFLoader reads a PDF, RecursiveCharacterTextSplitter chunks it (1000 chars, 200 overlap), OpenAI's text-embedding-3-large embeds each chunk, and QdrantVectorStore indexes the vectors into a Qdrant collection running via Docker.

This file only covers ingestion — there's no retrieval or answer-generation step yet, so it's honestly a partial pipeline rather than a working RAG app. The natural next step is a query-side script that embeds a question and retrieves against this same collection.

Uses the mem0 library configured with OpenAI for embeddings/LLM calls, Qdrant as the vector store, and Neo4j as a graph store for relational memory, wrapped around a basic chat loop.

This one is genuinely unfinished — there's a bug in how the OpenAI client reads its API key, so it's marked experimental rather than working. Worth noting: the API keys in this repo were originally hardcoded in plaintext and have since been moved to environment variables — a real fix, not just a portfolio talking point.

Minimal direct API calls to OpenAI's chat completions endpoint and Google's Gemini endpoint, run independently.

The request/response shape differs enough between OpenAI and Gemini that provider-agnostic code needs a thin abstraction layer — which LangChain is largely solving in the RAG script above.

Encodes and decodes sample text using the gpt-4o tokenizer and inspects vocabulary size and token IDs directly.

Token count isn't word count — seeing the actual integer IDs made prompt-length and cost estimation concrete instead of abstract.