Posts tagged: agentic AI

When I first set out to build MIKAI, my goal was simple: a personal AI assistant capable of managing medical knowledge, learning from interactions, and providing intelligent responses in Thai and English. But achieving that goal demanded more than just a large language model — it required memory, context, reasoning, and the ability to pull in external knowledge when needed. That’s where the Retrieval-Augmented Generation (RAG) methodology came in.

The Early Days: Memory Without Structure

In the beginning, MIKAI relied on basic local memory and a single LLM. The model could answer questions based on its training, but it struggled with continuity across sessions and nuanced technical queries. I realized that without a structured way to recall prior conversations and integrate external sources, MIKAI would hallucinate or repeat mistakes.

The first iteration used a Postgres database with pgvector for storing embeddings of past interactions. Every user query was embedded, and cosine similarity was used to pull semantically similar prior exchanges. This approach gave MIKAI a sense of continuity — it could “remember” previous sessions — but there were limitations. Embeddings alone cannot capture subtle medical nuances, and context retrieval often included irrelevant or redundant information.

Introducing the RAG Pipeline

To address these challenges, we implemented a full RAG pipeline. At its core, MIKAI now uses a hybrid system: a combination of local memory (Postgres/pgvector) and external knowledge bases (via Qdrant) to provide answers grounded in both past experience and curated content.

The pipeline begins with Query Preprocessing. Using front_llm.sharpen_query(query), MIKAI cleans and rewrites incoming questions while detecting the user’s language. This ensures that ambiguous queries are clarified before retrieval.

Next comes Embedding + Memory Retrieval. The sharpened query is converted into a vector embedding (self.embeddings.embed_query) and compared against session and global memory using memory_manager.retrieve_similar(). This allows MIKAI to fetch the most semantically relevant past interactions.

For external knowledge, Retriever Manager queries Qdrant collections based on keywords and context. For instance, if a user asks about a rare endocrine disorder, MIKAI identifies the appropriate collection (data_kb, hospital guidelines, research articles) and retrieves top-matching documents. Deduplication ensures that the top-3 documents are formatted into concise snippets for context fusion.

Context Fusion and Professor Mode

A crucial innovation in MIKAI is Context Fusion. Instead of simply concatenating memory and external documents, the system merges:

• Previous bot responses and user turns from local memory.
• Retrieved documents from Qdrant.
• Optional condensed summaries generated via memory_manager.condense_memory().

This combined context then enters Professor Mode, an extra reasoning layer (llm_manager.run_professor_mode()) where the model structures and interprets the context before generating a final answer. This step ensures that MIKAI doesn’t just regurgitate text but synthesizes a coherent response grounded in all available knowledge.

Finally, LLM Answer Generation (llm_manager.generate_rag_response) produces the answer. Clean-up steps remove repeated phrases, and optional back-translation ensures consistency if the query is not in English. If local memory or external knowledge fails to provide sufficient context, MIKAI can run a Web Search Fallback via DuckDuckGo, integrating the results into a regenerated answer.

Strengths of MIKAI’s RAG Approach

This pipeline has several notable strengths:

• Dual Memory System: By combining local memory with external knowledge bases, MIKAI balances continuity with factual accuracy.
• Condensation Step: Reduces irrelevant context and prevents context overflow in long conversations.
• Professor Mode: Adds reasoning and structure, transforming raw data into coherent, context-aware answers.
• Web Fallback: Ensures coverage even when the knowledge base lacks specific information.
• Importance Scoring & Scopes: Allows prioritization of critical knowledge over less relevant information.

These features make MIKAI more robust than a standard LLM and help maintain reliability in medical or technical domains.

Challenges and Limitations

Despite these strengths, the current system isn’t perfect:

• Embedding-Only Retrieval: Cosine similarity can drift for nuanced queries, potentially retrieving partially relevant memories.
• Echoing Past Mistakes: Using prior bot answers as context can propagate errors.
• Context Injection Gaps: generate_rag_response() currently seems to receive only the query, not the fully curated context, which may bypass context fusion benefits.
• Shallow Deduplication: Only compares first 200 characters of documents, risking subtle repetition.
• No Re-Ranking Across Sources: Memory and KB results are joined but not scored against each other for relevance.

Addressing these limitations will require passing the final fused context into the generation step, adding a re-ranking layer (e.g., BM25 or cross-encoder), and separating bot memory from external documents to prevent hallucinations.

MIKAI RAG in Practice

In practical use, MIKAI’s RAG system allows multiturn medical consultations, Thai-English language support, and intelligent reasoning over both past interactions and curated external knowledge. A patient can ask about leg edema, for example, and MIKAI retrieves previous session history, relevant hospital documents, and research articles, fusing them into a coherent explanation. If needed, it can augment its answer with a web search.

This pipeline has also enabled continuous learning. Every interaction is stored with embeddings and metadata (session/global/correction), allowing MIKAI to refine its memory, track repetition, and avoid redundant or low-quality responses over time.

The Road Ahead

Looking forward, the next steps for MIKAI involve:

• Ensuring final context injection into the generation step.
• Adding cross-source re-ranking to select the most relevant information.
• Improving deduplication and similarity scoring.
• Expanding external knowledge integration beyond Qdrant to include specialized medical databases and real-time research feeds.

The goal is to make MIKAI a fully reliable, continuously learning assistant that synthesizes knowledge across multiple modalities and timeframes.

Conclusion

From its early days as a simple memory-enabled LLM to today’s RAG-powered, professor-mode-enhanced assistant, MIKAI’s journey reflects the evolution of AI beyond static knowledge. By combining embeddings, vector databases, context fusion, reasoning layers, and web fallback, MIKAI demonstrates how a thoughtful RAG system can transform an LLM into a domain-aware, multiturn, multilingual assistant.

While challenges remain — especially around context injection and re-ranking — the framework is robust enough to provide continuity, accuracy, and intelligent reasoning in complex domains like medicine. As MIKAI continues to evolve, it promises to become an indispensable companion for knowledge work, patient consultation, and dynamic learning.

doctornuke

A web newbie since 1996

doctornuke.com/