— AI Engineer · Bengaluru, India

TUSHAR

▍

I build LLM agents, RAG systems and eval harnesses — in production at KPMG, and in the open, one commit at a time.

↓
…commits
…repositories
…pull requests
81/81eval suites passing

Projects

Open-source systems I design, build and measure. Every one ships with evals — each card links the repo and a live demo.

📄 Research note: how pattern-based guardrails fail — the mutation-based adversarial benchmark of the agent-service detector, scored on attack success, false refusal and completion.

📄 Research note: retrieval is not a safety boundary — the poisoned-corpus and unsupported-question benchmark of the rag-service pipeline, scored on attack success, fabrication, false refusal and completion.

📄 Research note: the gate held, the draft leaked — the three-metric evaluation of the support-agent's safety layers: zero unauthorised refunds across 171 attack runs, poisoned-KB quoting as the open channel, and a 31.2% false-refusal rate from the refund-keyword bug shape.

📄 Research note: the gate is a vocabulary — the three-metric evaluation of a voice agent's spoken confirmation step: a six-pattern yes vocabulary books contradictory and third-party confirmations, refuses natural ones like "sounds good" (35.3% false-refusal rate), and lets a bystander's yes book — with 33 of 137 runs crashing on KeyError 'idle'.

📄 Research note: citation is not truth — the three-metric evaluation of an extractive research pipeline: source-borne payloads succeed by construction (84.6% of seeds, 100% of variants) while citation accuracy stays 100% — because citations verify quotes, not claims.

📄 Research note: the judge reads tokens — the three-metric evaluation of an eval toolkit's own scorers: 62.5% of seed attacks pass while half the honest answers fail, and judge-vs-human agreement collapses 0.917 to 0.200 — a wrong answer scores a perfect 1.000 through a smuggled CASE block.

Stack

The tools I reach for in production.

LLM & Agents

LangChainLangGraphGoogle ADKLlamaIndexAutoGenCrewAIMCPOpenAI APIAnthropic APIGemini APIAzure OpenAIHugging FaceTransformersvLLMOllama

Retrieval & Data

Azure AI SearchPineconeQdrantWeaviateMilvuspgvectorFAISSElasticsearchEmbeddingsHybrid retrievalRerankingSQLPandasNumPy

Serving & MLOps

PythonFastAPIUvicornDockerKubernetesGitHub ActionsPytestPydanticREST APIsLinuxGitMLflowWeights & Biases

Practices

Eval-driven developmentRAG evalsPrompt engineeringStructured extractionLLM-as-judgeReAct workflowsGuardrails

Experience

Where I've been, and what I shipped along the way.

Open source — every day

2026 – Present

Four flagship systems built in the open — RAG, agents, LLM serving, PR review — each evolving with a real commit daily.

Microsoft Azure AI-102

2025 · Certified

Azure AI Engineer Associate — designing and implementing Azure AI solutions.

About

AI Engineer with 2 years shipping production GenAI at KPMG (Sep 2024 – present) — multi-agent pipelines, RAG over large document corpora, vector search at scale, and LoRA/QLoRA fine-tuning.

I build eval-first: every system ships with numbers, not vibes. And I open-source something new every single day.

Azure AI-102 certified (2025) · Based in Bengaluru, India

Contact

If you're hiring for AI engineering — or just want to talk agents and evals.

maxtushar292001@gmail.com GitHub ↗