MODULE 13 — RAG FUNDAMENTALS

6

13.1 What is RAG?

RAG stands for Retrieval-Augmented Generation.

It is an AI technique in which an AI system first retrieves relevant information from external sources and then uses that information to generate a more accurate and context-aware response.

RAG has three main stages:

R — Retrieval

The system searches external sources such as:

• Databases
• Documents
• Websites
• Knowledge bases
• PDFs
• Vector databases

A — Augmented

The retrieved information is added to the AI's context, providing relevant and potentially up-to-date knowledge.

G — Generation

The Large Language Model (LLM) uses the retrieved information together with its trained knowledge to generate the final answer.

Simple Flow

User Question → Retrieve Information → Add Context → Generate Answer

13.2 Why is RAG Important?

6

A normal LLM can face several problems:

• Outdated information
• Hallucinations or fabricated answers
• Lack of company-specific knowledge
• Poor performance on niche topics
• No direct access to recent information

RAG addresses these limitations by retrieving relevant external information before generating an answer.

Benefits of RAG

RAG can provide:

• More factually grounded responses
• Access to recent information
• Domain-specific knowledge
• Reduced hallucinations
• Personalised responses
• Greater reliability in production systems

13.3 How RAG Works — Step by Step

5

Step 1 — User Query

The user asks a question.

Example:

“What are the latest AI trends in healthcare?”

Step 2 — Information Retrieval

The retriever searches relevant sources such as:

• Research papers
• Company databases
• Knowledge documents
• Web pages

Step 3 — Context Injection

Relevant passages are inserted into the prompt to give the LLM targeted context.

Step 4 — AI Response Generation

The LLM processes the enriched prompt and generates a detailed response based on the retrieved information.

13.4 Architecture of a RAG System

6

A basic RAG architecture contains:

1. User Interface

The place where users submit their queries.

Input can include:

• Text
• Voice
• Images

2. Retriever

Searches knowledge stores and finds relevant content.

3. Vector Database

Stores document embeddings.

Examples mentioned in the course:

• Pinecone
• Weaviate
• ChromaDB

4. Embedding Model

Converts text into numerical vector representations.

5. LLM / Generator

Uses the retrieved information to generate the final response.

13.5 Embeddings and Vector Databases

5

What are Embeddings?

Embeddings are numerical representations of text that allow AI systems to measure semantic similarity.

Unlike simple keyword matching, embeddings help identify concepts with similar meanings.

Example

Dog ↔ Puppy

Car ↔ Automobile

Even though the words are different, they can have similar semantic relationships.

13.6 Types of Retrieval

Keyword Search

Looks for exact words within documents.

Advantage: Fast.

Limitation: May miss synonyms and contextual meaning.

Semantic Search

Uses embeddings to understand meaning and intent.

It can handle:

• Paraphrases
• Similar meanings
• Related concepts

Hybrid Search

Combines:

Keyword Search + Semantic Search

This aims to provide better accuracy, speed, and relevance.

13.7 Applications of RAG

7

Education

• AI tutors
• Smart study assistants
• Textbook-based Q&A
• Exam preparation

Business

• Customer support
• Internal knowledge assistants
• HR policy Q&A
• Automated reports

Healthcare

• Medical information retrieval
• Drug interaction lookup
• Clinical decision support
• Patient history analysis

Legal

• Contract analysis
• Legal document search
• Case-law research
• Compliance checking

13.8 Advantages and Challenges

Advantages

• Updated information
• Reduced hallucinations
• Better domain-specific knowledge
• Context-aware responses
• Improved factual accuracy
• Enterprise scalability

Challenges

• Complex architecture
• Need for high-quality databases
• Retrieval quality affects answer quality
• Additional computational costs
• Added retrieval latency
• Documents need to remain fresh

13.9 Future of RAG

The course highlights future developments such as:

• Faster and more accurate retrieval
• Multimodal RAG for images, audio and video
• Real-time enterprise AI
• Personalised AI experiences
• Agentic RAG
• Integration with IoT and edge computing