AyurIP-Sahayak
A research assistant for Ayurveda, traditional knowledge and Indian IP law. Ask in English, हिन्दी, ಕನ್ನಡ or Hinglish; it answers only from its sources and cites the exact page behind every claim.
0.99answer faithfulness0.95retrieval Hit@14 languagesEnglish, Hindi, Kannada, Hinglish
Read the full write-up architecture, method, results, what broke
How it works
- Retrieval written by hand. Section-aware PDF chunks, Gemini embeddings and NumPy cosine search, plus BM25 and Reciprocal Rank Fusion built from scratch to compare against.
- Decisions from data. Hybrid search is the usual advice. On 38 labelled questions it lowered MRR from 0.97 to 0.93, so dense search stayed the default.
- Answers judged, and the judge tested. A model from a different family checks every answer claim against its passages. I planted 15 false sentences to test it; it caught all 15.
Built with Python, FastAPI, Google Gemini API, NumPy, React, Docker, GitHub Actions, and pytest (47 tests).
Results
| Measure | Result |
|---|---|
| Answer faithfulness166 of 167 claims supported | 0.99 |
| Citation precisioncited passage backs the claim | 0.98 |
| Retrieval MRR0.93 with hybrid search | 0.97 |
| Off-topic questions refusedsays "not covered" instead of guessing | 5/5 |
What broke, and how I fixed it
-
01
Down to 5 runtime packages
The first deploy ran out of memoryAyurIP
A local PyTorch embedding model went past the free server's 512 MiB. I moved to an embeddings API and plain NumPy search.
-
02
MRR 0.97 vs 0.93
The "best practice" made search worseAyurIP
Fusing in keyword search pushed correct passages down a rank or two. Dense search stayed the default; hybrid is one setting away.
-
03
4/15 → 0/15
Kannada answers showed broken lettersAyurIP
The model slipped Devanagari letters into Kannada words. I mapped them across Unicode blocks (they sit 0x380 apart), with tests.
-
04
15/15 correct
English questions came back in KannadaAyurIP
A Kannada example in my own prompt pulled the model off course. The code now picks the reply language and states it in the prompt.