What Is MCP (Model Context Protocol)? A Beginner’s Guide

AI applications are becoming more powerful, but an LLM by itself has an important limitation: it cannot automatically access every external system, database, file, or application.

This is where MCP comes into the picture.

MCP stands for Model Context Protocol. It is an open protocol designed to standardize how AI applications connect models to external tools, data sources, and capabilities.

In this beginner-friendly guide, we will understand what MCP is, why it is needed, how MCP works, its architecture, MCP servers and clients, tools, resources, prompts, and how MCP fits into modern AI agents.

What Is MCP?

MCP (Model Context Protocol) is a standardized way for AI applications to interact with external systems.

Instead of building a completely different integration for every AI application, an MCP-based architecture provides a common protocol for exposing capabilities to AI clients.

A simplified view is:

AI Application
      ↓
   MCP Client
      ↓
   MCP Server
      ↓
 ┌────┼───────────┐
 ↓    ↓           ↓
Files Database   APIs

This allows an AI application to discover and use capabilities exposed by MCP servers.

Why Was MCP Needed?

Imagine you are building an AI assistant that needs access to:

• Company documents
• Git repositories
• Databases
• File systems
• Project management systems
• Customer information
• External APIs

Without a standardized approach, each integration may require custom code.

AI Application
   ├── Custom Git Integration
   ├── Custom Database Integration
   ├── Custom File Integration
   ├── Custom API Integration
   └── Custom CRM Integration

As the number of integrations increases, the architecture becomes harder to maintain.

MCP provides a standardized protocol that can simplify these integrations.

MCP in Simple Terms

A simple way to think about MCP is:

MCP = A standard way for AI applications
      to discover and use external capabilities.

For example, an MCP server could expose a tool called:

search_documents

The AI application can discover that tool and invoke it when appropriate.

MCP Architecture

A basic MCP architecture contains three important components:

• MCP Host
• MCP Client
• MCP Server

The relationship can be visualized as:

                 MCP Host
              AI Application
                    │
                    │
                MCP Client
                    │
                    │
              MCP Protocol
                    │
                    ↓
                MCP Server
              ┌─────┼─────┐
              ↓     ↓     ↓
            Tools Resources Prompts

What Is an MCP Host?

The MCP Host is the AI application that wants to use MCP capabilities.

Examples could include an AI-powered desktop application, IDE, or another application that supports MCP.

The host manages the overall interaction with the user and the AI model.

User
 ↓
MCP Host
 ↓
LLM
 ↓
MCP Client

What Is an MCP Client?

An MCP Client is the component responsible for communicating with an MCP server.

The client maintains the protocol connection and handles communication between the host application and the server.

MCP Host
   ↓
MCP Client
   ↓
MCP Server

A host can use MCP clients to connect to one or more MCP servers.

What Is an MCP Server?

An MCP Server exposes capabilities that an MCP client can discover and use.

For example, an MCP server could provide:

• A file search tool
• A database query tool
• A Git repository tool
• Company documentation resources
• Custom application operations

Conceptually:

MCP Server
    │
    ├── Tool: searchFiles
    ├── Tool: getCustomer
    ├── Tool: executeQuery
    │
    ├── Resource: company-docs
    └── Prompt: support-assistant

What Are MCP Tools?

Tools represent actions that an AI application can invoke through an MCP server.

For example:

Tool:
search_customer

Input:
{
    "customerId": "12345"
}

Output:
Customer information

Another tool might be:

Tool:
search_products

Input:
{
    "query": "wireless keyboard"
}

The important concept is that the AI application can discover the available tool and its input requirements.

What Are MCP Resources?

Resources represent data or contextual information that can be made available through an MCP server.

For example:

• Documentation
• Files
• Database information
• Application configuration
• Other contextual data

Conceptually:

MCP Server
     ↓
Resources
     ├── Documentation
     ├── Configuration
     └── Application Data

Resources are different from tools because a tool represents an operation, while a resource represents information that can be accessed as context.

What Are MCP Prompts?

Prompts allow an MCP server to expose reusable prompt templates or workflows.

For example:

Prompt:
analyze_support_ticket

Arguments:
ticketId
customerId

Purpose:
Analyze a customer support ticket
and produce a structured summary.

This can help standardize how particular tasks are performed.

MCP Tools vs Resources vs Prompts

Component	Purpose	Example
Tools	Perform actions	Search database
Resources	Provide information	Read documentation
Prompts	Provide reusable prompt templates	Analyze support ticket

How Does MCP Work?

Let's consider a simple example.

A user asks an AI assistant:

"Find the latest information about
customer 10025."

The AI determines that it needs information from an external customer system.

User Question
      ↓
      LLM
      ↓
Need Customer Information
      ↓
MCP Client
      ↓
MCP Server
      ↓
Customer Tool
      ↓
Customer System
      ↓
Result
      ↓
LLM
      ↓
Final Answer

MCP Tool Discovery

One of the important ideas in MCP is that clients can discover what capabilities an MCP server exposes.

For example, an MCP server might expose:

Available Tools

1. search_customer
2. get_customer_orders
3. search_products
4. create_support_ticket

The AI application can use the tool definitions to understand what operations are available and what inputs they require.

MCP and Function Calling

If you have worked with LLM APIs, you may already know about function calling or tool calling.

Function calling allows a model to request that an application execute a function.

For example:

LLM
 ↓
"Call get_customer"
 ↓
Application
 ↓
Customer API
 ↓
Result
 ↓
LLM

MCP is broader than simply defining a function.

It provides a standardized protocol and interaction model for exposing tools, resources, and prompts to compatible AI applications.

MCP vs Traditional API

A traditional API might expose endpoints such as:

GET /api/customers/10025
GET /api/orders/10025
POST /api/support/tickets

An MCP server could expose corresponding capabilities in a way that MCP-compatible clients can discover and interact with.

MCP Server

get_customer
get_customer_orders
create_support_ticket

The underlying implementation could still call your existing APIs.

AI Application
      ↓
MCP
      ↓
MCP Server
      ↓
Existing REST API
      ↓
Business System

This means MCP does not necessarily replace your existing APIs.

MCP Can Sit on Top of Existing Systems

This is particularly useful for enterprise applications.

Suppose your company already has:

POS API
Order API
Customer API
Inventory API
Database

An MCP server can provide AI-friendly capabilities on top of these systems.

                AI Application
                      ↓
                   MCP Client
                      ↓
                  MCP Server
                ┌─────┼─────┐
                ↓     ↓     ↓
              POS   Orders Customers
               API    API     API

The existing business systems do not necessarily need to become AI systems themselves.

MCP and AI Agents

MCP becomes particularly interesting when building AI agents.

An AI agent may need to:

• Read files
• Search information
• Query databases
• Call APIs
• Execute business operations
• Use multiple tools

MCP provides a standardized way for compatible AI applications to discover and use those capabilities.

                 AI Agent
                     ↓
                MCP Client
                     ↓
       ┌─────────────┼─────────────┐
       ↓             ↓             ↓
   MCP Server A  MCP Server B  MCP Server C
       ↓             ↓             ↓
     Files        Database       APIs

Example: AI Developer Assistant

Imagine building an AI assistant for software developers.

The assistant might need:

• Git repository access
• File search
• Documentation search
• Issue tracking
• Database access

Instead of implementing every integration directly inside the AI application, MCP servers can expose these capabilities.

Developer
    ↓
AI Coding Assistant
    ↓
MCP Client
    ↓
 ┌──────────┬──────────┬───────────┐
 ↓          ↓          ↓
Git MCP   Docs MCP   Database MCP
 ↓          ↓          ↓
Git       Docs       Database

Example: AI Customer Support Agent

Consider a customer support application.

The AI agent needs to:

Search Customer
      ↓
Check Orders
      ↓
Check Product
      ↓
Create Support Ticket

An MCP server could expose these operations as tools:

search_customer
get_orders
search_product
create_ticket

The AI agent can decide which tool is appropriate based on the user's request.

MCP Request and Response

MCP communication is based on a structured protocol. MCP uses JSON-RPC 2.0 messages for protocol communication.

A simplified request can look conceptually like:

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "search_customer",
    "arguments": {
      "customerId": "10025"
    }
  }
}

The server processes the request and returns a structured result.

The exact protocol messages and capabilities depend on the MCP specification and implementation.

MCP Transport

MCP communication requires a transport mechanism between the client and server.

Depending on the deployment scenario, MCP implementations can use supported transports appropriate to local or remote communication.

For example:

Local Application
      ↓
Local MCP Server

Remote Application
      ↓
Network
      ↓
Remote MCP Server

The transport layer is separate from the higher-level MCP concepts such as tools and resources.

MCP Security

Security is extremely important when AI applications can access external systems.

Imagine an AI assistant has access to a database and a tool that can modify customer records.

You should carefully control:

• Which tools are available
• Which users can access them
• What arguments are allowed
• Which resources can be accessed
• Authentication
• Authorization
• Audit logging
• Data privacy

For example:

User
 ↓
Authentication
 ↓
Authorization
 ↓
MCP Client
 ↓
MCP Server
 ↓
Tool
 ↓
Business System

An MCP server should not automatically expose every internal operation to an AI model.

Read-Only vs Write Tools

There is an important difference between read and write operations.

A read-only tool might be:

get_customer()
search_document()
get_order()

A write operation might be:

create_order()
update_customer()
delete_file()

Write operations can have real-world consequences and should therefore be protected with appropriate authorization, validation, confirmation, and auditing.

MCP vs RAG

MCP and RAG are related to AI applications, but they solve different problems.

RAG	MCP
Retrieves relevant information	Standardizes connections to external capabilities
Commonly uses embeddings and vector search	Can expose tools, resources, and prompts
Useful for knowledge retrieval	Useful for connecting AI applications to systems
Often retrieves documents	Can enable actions as well as access to information

They can also be combined.

AI Agent
   ↓
MCP
   ↓
RAG Tool
   ↓
Vector Database
   ↓
Relevant Documents
   ↓
LLM

MCP vs API

API	MCP
General software integration mechanism	Protocol designed for AI application context and capabilities
Usually designed around application-specific contracts	Provides standardized AI-oriented interactions
Clients typically need API-specific knowledge	MCP clients can discover supported capabilities
Can be used by any software	Designed for compatible AI applications and servers

MCP does not make traditional APIs obsolete. In many architectures, an MCP server can actually use existing APIs behind the scenes.

MCP vs Function Calling

Function Calling	MCP
Model requests a function/tool invocation	Standardized protocol for AI-to-capability interaction
Often implemented inside a specific application	Designed for reusable integrations
Tool definitions are usually application-specific	Capabilities can be exposed through MCP servers

Why MCP Is Important for Developers

MCP can change how developers think about AI integrations.

Instead of building:

AI App
  ↓
Custom Database Code

AI App
  ↓
Custom Git Code

AI App
  ↓
Custom File Code

AI App
  ↓
Custom API Code

You can build standardized MCP servers around reusable capabilities.

                AI Applications
                 /     |      \
                /      |       \
               ↓       ↓        ↓
            MCP Client(s)
                 ↓
        Standard MCP Protocol
                 ↓
      ┌──────────┼──────────┐
      ↓          ↓          ↓
   MCP Server MCP Server MCP Server
      ↓          ↓          ↓
    Git       Database     Files

Example MCP Server for a .NET Developer

As a .NET developer, you could create an MCP server that exposes capabilities from an existing .NET application.

For example:

ASP.NET Core Application
          ↓
      Business Layer
          ↓
      MCP Server
          ↓
 ┌────────┼─────────┐
 ↓        ↓         ↓
Customers Orders  Inventory

The MCP server could expose tools such as:

get_customer
search_orders
check_inventory
search_product

An AI application that supports MCP could then discover these capabilities.

Example Enterprise Architecture

A larger enterprise architecture could look like:

                         User
                           ↓
                      AI Assistant
                           ↓
                         LLM
                           ↓
                      MCP Client
                           ↓
                   MCP Protocol
                           ↓
        ┌──────────────────┼──────────────────┐
        ↓                  ↓                  ↓
    Customer MCP       POS MCP           Docs MCP
        ↓                  ↓                  ↓
 Customer API          POS API        Vector Database
        ↓                  ↓                  ↓
     Customer            POS             Documents

This architecture separates the AI application from the implementation details of each external system.

Can MCP Replace RAG?

No.

MCP and RAG operate at different levels.

MCP can expose a RAG capability as a tool or resource.

AI Agent
   ↓
MCP Client
   ↓
RAG MCP Server
   ↓
Embedding
   ↓
Vector Search
   ↓
Documents

In this example, MCP provides the standardized connection while RAG performs information retrieval.

Can MCP Replace APIs?

Not necessarily.

Traditional APIs remain useful for application-to-application communication.

MCP can provide an AI-oriented interface on top of existing services.

AI Application
      ↓
MCP
      ↓
MCP Server
      ↓
REST API
      ↓
Existing Application

MCP and the Future of AI Applications

Modern AI applications are moving beyond simple question-and-answer interfaces.

AI systems increasingly need to:

• Retrieve information
• Use tools
• Access external systems
• Perform multi-step tasks
• Interact with business applications

This is where protocols such as MCP become particularly useful.

LLM
 ↓
Reason
 ↓
Select Tool
 ↓
MCP
 ↓
External System
 ↓
Tool Result
 ↓
Reason Again
 ↓
Final Answer

This pattern is one of the building blocks behind modern AI agents.

Common MCP Terminology

Term	Meaning
MCP	Model Context Protocol
MCP Host	AI application that uses MCP
MCP Client	Component that communicates with an MCP server
MCP Server	Server that exposes capabilities
Tool	Action that can be invoked
Resource	Information or contextual data
Prompt	Reusable prompt template or workflow
JSON-RPC	Message format used by MCP protocol communication

Key Takeaways

• MCP stands for Model Context Protocol.
• MCP provides a standardized way for compatible AI applications to interact with external capabilities.
• An MCP architecture commonly involves a host, client, and server.
• MCP servers can expose tools, resources, and prompts.
• Tools allow AI applications to perform operations.
• Resources provide contextual information.
• Prompts can provide reusable prompt templates.
• MCP can work with existing APIs and business systems.
• MCP and RAG solve different problems and can be used together.
• MCP is particularly useful when building AI assistants and agents that interact with external systems.
• Security, authorization, validation, and auditing are important when exposing powerful tools.

Conclusion

MCP is an important building block for connecting AI applications to the outside world.

An LLM can generate text and reason over information, but useful AI applications often need access to documents, databases, APIs, files, and business systems.

MCP provides a standardized protocol for exposing these capabilities to compatible AI applications.

The basic architecture can be remembered as:

AI Application
      ↓
   MCP Client
      ↓
   MCP Server
      ↓
Tools / Resources / Prompts
      ↓
External Systems

Once you understand MCP, the next logical step is to understand how it compares with traditional APIs and function calling, and then see how MCP can be used to build an actual AI Agent.

Next: MCP vs APIs vs Function Calling: What's the Difference? — Learn how these three approaches differ and when a developer should use each one.

Build a RAG Application with C# and .NET 8

In the previous articles, we explored RAG, embeddings, vector databases, semantic search, and fine-tuning.

Now let's build something practical.

In this tutorial, we will build a simple RAG (Retrieval-Augmented Generation) application using C# and .NET 8.

The application will allow an AI model to answer questions using information from our own documents instead of relying only on the model's built-in knowledge.

We will use Ollama to run an LLM locally and build the RAG pipeline using C#.

What We Are Going to Build

Our application will follow this architecture:

                Document
                    ↓
              Text Extraction
                    ↓
                 Chunking
                    ↓
             Embedding Model
                    ↓
              Vector Storage
                    ↓
              Semantic Search
                    ↓
             Relevant Chunks
                    ↓
                    LLM
                    ↓
                 Answer

For example, imagine we have a company document containing:

Employees are entitled to 20 days
of paid annual leave every year.

Unused leave can be carried forward
according to company policy.

The user can ask:

How many annual leave days do employees get?

The RAG application retrieves the relevant document content and provides it to the LLM.

Technologies Used

• C#
• .NET 8
• Ollama
• LLM
• Embedding Model
• Vector Search

To keep the example easy to understand, we will implement a lightweight vector store directly in C# rather than introducing a full production vector database.

The same architecture can later be extended to databases such as Qdrant, PostgreSQL with pgvector, Elasticsearch, Azure AI Search, or other vector databases.

Prerequisites

Before starting, install:

• .NET 8 SDK
• Ollama
• A local LLM model
• An embedding model

Verify the .NET installation:

dotnet --version

You should see a .NET SDK version beginning with 8 or a compatible newer SDK.

1. Install Ollama

Ollama allows you to run supported AI models locally.

After installing Ollama, verify that it is available:

ollama --version

Start an LLM model. For example:

ollama pull llama3.2

Then run it:

ollama run llama3.2

You can use another compatible model depending on your hardware.

2. Download an Embedding Model

RAG requires an embedding model to convert documents and user questions into vectors.

For example:

ollama pull nomic-embed-text

The embedding model and chat model serve different purposes.

Model	Purpose
Embedding Model	Converts text into vectors
LLM	Generates the final answer

3. Create a .NET 8 Project

Create a new console application:

dotnet new console -n SimpleRag
cd SimpleRag

Open the project in Visual Studio or your preferred IDE.

4. Understand the RAG Pipeline

Before writing code, let's understand the complete flow.

             INGESTION

Documents
    ↓
Split into Chunks
    ↓
Generate Embeddings
    ↓
Store Vectors


             QUERY

User Question
    ↓
Generate Query Embedding
    ↓
Similarity Search
    ↓
Retrieve Relevant Chunks
    ↓
Send Context to LLM
    ↓
Generate Answer

There are therefore two major stages:

• Ingestion: Prepare and index the documents.
• Query: Retrieve relevant information and generate an answer.

5. Create Sample Documents

For this tutorial, create a folder called Documents.

Create a file called company-policy.txt:

Company Leave Policy

Employees receive 20 days of paid annual leave every year.

Employees should submit leave requests through the employee portal.

Unused annual leave can be carried forward according to the company's leave policy.

Managers are responsible for approving leave requests.

Emergency leave should be reported to the manager as soon as possible.

6. Create a Document Chunk

A document is usually divided into smaller pieces before generating embeddings.

Create a simple class:

public class DocumentChunk
{
    public string Text { get; set; } = string.Empty;

    public float[] Embedding { get; set; } = Array.Empty<float>();
}

Each chunk contains:

• The original text
• The embedding vector

7. Create the Ollama Embedding Method

Ollama exposes an HTTP API that can be called from C#.

Create a method to generate an embedding:

using System.Net.Http.Json;

public class OllamaEmbeddingService
{
    private readonly HttpClient _httpClient;

    public OllamaEmbeddingService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<float[]> GenerateEmbeddingAsync(string text)
    {
        var request = new
        {
            model = "nomic-embed-text",
            prompt = text
        };

        var response = await _httpClient.PostAsJsonAsync(
            "http://localhost:11434/api/embeddings",
            request);

        response.EnsureSuccessStatusCode();

        var result =
            await response.Content.ReadFromJsonAsync<EmbeddingResponse>();

        return result?.Embedding ?? Array.Empty<float>();
    }

    private class EmbeddingResponse
    {
        public float[] Embedding { get; set; } = Array.Empty<float>();
    }
}

The embedding model converts the supplied text into a numerical vector.

8. What Does an Embedding Look Like?

You don't normally need to understand every number in an embedding.

Conceptually, the result looks like:

"Employees receive 20 days of annual leave."

        ↓

[0.023, -0.451, 0.812, 0.117, ...]

The vector can contain hundreds or thousands of dimensions depending on the embedding model.

9. Create a Simple Text Chunker

For a production application, chunking requires more sophisticated rules.

For this tutorial, we can use paragraphs as chunks:

public static List<string> SplitIntoChunks(string text)
{
    return text
        .Split(
            new[] { "\r\n\r\n", "\n\n" },
            StringSplitOptions.RemoveEmptyEntries)
        .Select(x => x.Trim())
        .Where(x => !string.IsNullOrWhiteSpace(x))
        .ToList();
}

This keeps the example simple.

10. Create a Simple Vector Store

For demonstration purposes, we can keep the vectors in memory.

public class VectorStore
{
    private readonly List<DocumentChunk> _chunks = new();

    public void Add(DocumentChunk chunk)
    {
        _chunks.Add(chunk);
    }

    public List<DocumentChunk> GetAll()
    {
        return _chunks;
    }
}

In a real application, the vectors would usually be persisted in a vector database.

11. Calculate Cosine Similarity

To determine how similar two vectors are, we can use cosine similarity.

The basic idea is:

Query Vector
     ↓
Compare with Document Vector
     ↓
Similarity Score
     ↓
Rank Results

Create a helper method:

public static double CosineSimilarity(
    float[] vectorA,
    float[] vectorB)
{
    if (vectorA.Length != vectorB.Length)
        throw new ArgumentException("Vector dimensions must match.");

    double dotProduct = 0;
    double magnitudeA = 0;
    double magnitudeB = 0;

    for (int i = 0; i < vectorA.Length; i++)
    {
        dotProduct += vectorA[i] * vectorB[i];

        magnitudeA += vectorA[i] * vectorA[i];
        magnitudeB += vectorB[i] * vectorB[i];
    }

    if (magnitudeA == 0 || magnitudeB == 0)
        return 0;

    return dotProduct /
           (Math.Sqrt(magnitudeA) * Math.Sqrt(magnitudeB));
}

A higher similarity score generally indicates that the vectors are more closely aligned according to this metric.

12. Implement Semantic Search

Now we can search our vector store.

public List<(DocumentChunk Chunk, double Score)> Search(
    float[] queryEmbedding,
    int topK = 3)
{
    return _chunks
        .Select(chunk => (
            Chunk: chunk,
            Score: CosineSimilarity(
                queryEmbedding,
                chunk.Embedding)))
        .OrderByDescending(x => x.Score)
        .Take(topK)
        .ToList();
}

This is the core of our semantic retrieval process.

13. Index the Documents

Now let's read the document and create embeddings for each chunk.

var httpClient = new HttpClient();

var embeddingService =
    new OllamaEmbeddingService(httpClient);

var vectorStore = new VectorStore();

var documentText =
    await File.ReadAllTextAsync(
        "Documents/company-policy.txt");

var chunks =
    SplitIntoChunks(documentText);

foreach (var chunkText in chunks)
{
    var embedding =
        await embeddingService
            .GenerateEmbeddingAsync(chunkText);

    vectorStore.Add(new DocumentChunk
    {
        Text = chunkText,
        Embedding = embedding
    });
}

At this point, our document has been converted into searchable vectors.

14. Generate an Embedding for the User Question

Suppose the user asks:

How many vacation days do employees get?

Generate an embedding for the question:

var question =
    "How many vacation days do employees get?";

var queryEmbedding =
    await embeddingService
        .GenerateEmbeddingAsync(question);

Now both the question and documents exist in the same embedding space.

15. Search for Relevant Documents

Perform a similarity search:

var results =
    vectorStore.Search(
        queryEmbedding,
        topK: 3);

foreach (var result in results)
{
    Console.WriteLine(
        $"Score: {result.Score:F4}");

    Console.WriteLine(
        result.Chunk.Text);

    Console.WriteLine();
}

The result might look conceptually like:

Score: 0.91

Employees receive 20 days of paid annual leave every year.

--------------------------------

Score: 0.74

Unused annual leave can be carried forward according to the company's leave policy.

--------------------------------

Score: 0.42

Managers are responsible for approving leave requests.

The highest-ranked chunks are the most relevant candidates for the answer.

16. Send the Retrieved Context to the LLM

Now we have the relevant information.

We need to provide that information to the LLM.

Create a prompt such as:

You are a helpful company policy assistant.

Answer the question using only the context provided below.

Context:
Employees receive 20 days of paid annual leave every year.

Question:
How many vacation days do employees get?

Answer:

The LLM can now generate an answer based on the retrieved context.

17. Call the Ollama Chat API

We can call the Ollama API using HttpClient.

public class OllamaChatService
{
    private readonly HttpClient _httpClient;

    public OllamaChatService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<string> GenerateAsync(
        string prompt)
    {
        var request = new
        {
            model = "llama3.2",
            prompt = prompt,
            stream = false
        };

        var response = await _httpClient.PostAsJsonAsync(
            "http://localhost:11434/api/generate",
            request);

        response.EnsureSuccessStatusCode();

        var result =
            await response.Content.ReadFromJsonAsync<OllamaResponse>();

        return result?.Response ?? string.Empty;
    }

    private class OllamaResponse
    {
        public string Response { get; set; } =
            string.Empty;
    }
}

18. Build the Final RAG Prompt

Combine the retrieved chunks:

var context = string.Join(
    "\n\n",
    results.Select(x => x.Chunk.Text));

var prompt = $"""
You are a helpful company policy assistant.

Answer the question using only the supplied context.

If the answer cannot be found in the context,
say that the information is not available.

Context:
{context}

Question:
{question}

Answer:
""";

This gives the LLM the information retrieved from our document.

19. Generate the Final Answer

Create the chat service:

var chatService =
    new OllamaChatService(httpClient);

var answer =
    await chatService.GenerateAsync(prompt);

Console.WriteLine("Question:");
Console.WriteLine(question);

Console.WriteLine();

Console.WriteLine("Answer:");
Console.WriteLine(answer);

The application should produce an answer similar to:

Employees receive 20 days of paid annual leave every year.

20. Complete RAG Flow

We have now implemented the complete basic RAG pipeline.

company-policy.txt
        ↓
Read Document
        ↓
Split into Chunks
        ↓
Generate Embeddings
        ↓
Store Vectors
        ↓
       ┌──────────────────┐
       │                  │
       │   User Question  │
       │                  │
       └────────┬─────────┘
                ↓
        Generate Embedding
                ↓
        Cosine Similarity
                ↓
          Top-K Chunks
                ↓
        Build RAG Prompt
                ↓
              Ollama
                ↓
          Generated Answer

21. Complete Example

The following example puts the main pieces together:

using System.Net.Http.Json;

var httpClient = new HttpClient();

var embeddingService =
    new OllamaEmbeddingService(httpClient);

var chatService =
    new OllamaChatService(httpClient);

var vectorStore =
    new VectorStore();

var documentText =
    await File.ReadAllTextAsync(
        "Documents/company-policy.txt");

var chunks =
    SplitIntoChunks(documentText);

foreach (var chunkText in chunks)
{
    var embedding =
        await embeddingService
            .GenerateEmbeddingAsync(chunkText);

    vectorStore.Add(new DocumentChunk
    {
        Text = chunkText,
        Embedding = embedding
    });
}

Console.Write("Ask a question: ");

var question =
    Console.ReadLine() ?? string.Empty;

var queryEmbedding =
    await embeddingService
        .GenerateEmbeddingAsync(question);

var results =
    vectorStore.Search(
        queryEmbedding,
        topK: 3);

var context =
    string.Join(
        "\n\n",
        results.Select(x => x.Chunk.Text));

var prompt = $"""
You are a helpful company policy assistant.

Answer the question using only the supplied context.

If the answer cannot be found in the context,
say that the information is not available.

Context:
{context}

Question:
{question}

Answer:
""";

var answer =
    await chatService.GenerateAsync(prompt);

Console.WriteLine();
Console.WriteLine("Answer:");
Console.WriteLine(answer);

static List<string> SplitIntoChunks(string text)
{
    return text
        .Split(
            new[] { "\r\n\r\n", "\n\n" },
            StringSplitOptions.RemoveEmptyEntries)
        .Select(x => x.Trim())
        .Where(x => !string.IsNullOrWhiteSpace(x))
        .ToList();
}

22. What Happens When the User Asks a Question?

Suppose the user asks:

Can I carry unused leave to the next year?

The application does not simply send the question directly to the LLM.

Instead:

User Question
      ↓
Embedding
      ↓
Vector Similarity Search
      ↓
Relevant Leave Policy Chunk
      ↓
RAG Prompt
      ↓
LLM
      ↓
Answer

This is the fundamental idea behind RAG.

23. Why Not Just Send the Entire Document to the LLM?

You might ask:

Why do we need embeddings and vector search?
Why not send the entire document to the LLM?

For small documents, that may be perfectly reasonable.

However, real applications can contain:

• Thousands of documents
• Millions of paragraphs
• Large technical manuals
• Customer records
• Frequently updated information

Sending everything to the LLM for every question is inefficient and may exceed the model's context capacity.

RAG retrieves only the information that is likely to be relevant.

24. Why Chunking Matters

Imagine a document contains 100 pages.

If the entire document is treated as one vector, the search may not be precise enough.

Instead:

100 Page Document
       ↓
Chunk 1
Chunk 2
Chunk 3
...
Chunk 500
       ↓
Embedding for each chunk
       ↓
Precise Retrieval

Good chunking can significantly affect the quality of a RAG system.

25. Improving the Chunking Strategy

The simple paragraph-based chunking used in this tutorial is only a starting point.

Production systems may use:

• Fixed token-size chunks
• Sentence-based chunks
• Paragraph-based chunks
• Heading-aware chunking
• Recursive chunking
• Overlapping chunks

For example:

Chunk 1:
Employees receive 20 days of annual leave...

Chunk 2:
...annual leave can be carried forward...

Overlap:
Some content appears in both chunks.

Overlap can help preserve context between chunks, although the appropriate strategy depends on the document type and retrieval requirements.

26. Why Use a Vector Database?

Our example keeps vectors in memory:

List<DocumentChunk>
```

This is fine for learning, but it is not a production-ready storage solution.

A real application may use a vector database to provide:


  
Persistent storage
  
Fast similarity search
  
Metadata filtering
  
Scalability
  
Indexing
  
Access control





27. Production RAG Architecture

A production architecture could look like this:

              Documents
                   ↓
            Document Processor
                   ↓
               Chunking
                   ↓
            Embedding Model
                   ↓
             Vector Database
                   │
                   │
                   ↓
             Retrieval API
                   ↑
                   │
              User Query
                   ↓
             Query Embedding
                   ↓
           Similarity Search
                   ↓
             Top-K Results
                   ↓
               Reranking
                   ↓
            Prompt Construction
                   ↓
                  LLM
                   ↓
                Answer




28. RAG With a Vector Database

Instead of:

C# List
   ↓
Cosine Similarity
   ↓
Results

A production system could use:

C#
 ↓
Vector Database
 ↓
Vector Search
 ↓
Top-K Results

The application code remains conceptually similar even when the underlying storage technology changes.




29. RAG Does Not Mean Only Vector Search

A mature RAG system can combine multiple retrieval techniques.

User Query
     ↓
 ┌───┴────────────┐
 ↓                ↓
Keyword       Semantic
Search        Search
 ↓                ↓
 └───────┬────────┘
         ↓
      Ranking
         ↓
      Reranking
         ↓
    Relevant Context
         ↓
        LLM

This approach is often called Hybrid Search.




30. Common RAG Problems

A RAG application can still produce poor answers.

Common causes include:


  
Poor document extraction
  
Bad chunking
  
Weak embeddings
  
Irrelevant retrieval results
  
Incorrect Top-K value
  
Missing metadata filters
  
Poor prompts
  
Insufficient context
  
Too much irrelevant context


Building a good RAG system is therefore more than simply adding a vector database.




31. Add a Relevance Threshold

You can prevent obviously irrelevant documents from being passed to the LLM.

var results = vectorStore.Search(
    queryEmbedding,
    topK: 5);

var relevantResults =
    results
        .Where(x => x.Score >= 0.70)
        .ToList();

The exact threshold should be determined through testing. A score such as 0.70 is only an example and should not be treated as a universal cutoff.




32. Add Source Information

A useful improvement is to store metadata along with each chunk.

public class DocumentChunk
{
    public string Text { get; set; } = string.Empty;

    public float[] Embedding { get; set; } =
        Array.Empty<float>();

    public string Source { get; set; } =
        string.Empty;

    public int ChunkNumber { get; set; }
}

This allows the application to tell the user where the information came from.




33. Example with Source Citations

The final answer could be displayed as:

Employees receive 20 days of paid annual leave every year.

Source:
company-policy.txt

For enterprise applications, source references can improve transparency and make it easier for users to verify the answer.




34. Security Considerations

Production RAG systems should also consider security.

For example, if a user does not have permission to view a document, the retrieval system should not return that document to the LLM.

User
 ↓
Authentication
 ↓
Authorization
 ↓
Metadata Filtering
 ↓
Vector Search
 ↓
Allowed Documents
 ↓
LLM

Access control should therefore be part of the retrieval architecture rather than added as an afterthought.




35. RAG vs Normal LLM Application


  

    
Normal LLM
    
RAG Application
  
  

    
User question
    
User question
  
  

    
Prompt
    
Generate query embedding
  
  

    
LLM
    
Search knowledge base
  
  

    
Answer
    
Provide retrieved context to LLM
  
  

    

    
Generate answer
  





36. Key Takeaways


  
RAG combines information retrieval with LLM generation.
  
Documents are normally divided into smaller chunks.
  
Embedding models convert chunks into vectors.
  
Vectors can be stored in a vector database or other vector index.
  
User questions are also converted into embeddings.
  
Similarity search retrieves relevant information.
  
The retrieved information is added to the LLM prompt.
  
The LLM generates the final response using the retrieved context.
  
Production systems usually require better chunking, persistent storage, filtering, evaluation, and security.





Conclusion

In this tutorial, we built a simple RAG application using C# and .NET 8.

We started with a document, divided it into chunks, generated embeddings using Ollama, stored the vectors in memory, performed semantic search using cosine similarity, and finally provided the retrieved context to an LLM.

The complete concept can be summarized as:

Documents
    ↓
Chunking
    ↓
Embeddings
    ↓
Vector Storage
    ↓
Semantic Search
    ↓
Relevant Context
    ↓
LLM
    ↓
Answer

This simple application demonstrates the core principles behind much larger production RAG systems.

Once you understand this pipeline, you can replace the in-memory vector store with a real vector database, add PDF processing, introduce hybrid search and reranking, and expose the RAG pipeline through an ASP.NET Core API.


  Next: In the next article, we will explore What Is MCP (Model Context Protocol)? and understand how AI models can interact with external tools, applications, and data sources through a standardized protocol.

RAG vs Fine-Tuning: What’s the Difference and When Should You Use Each?

In the previous articles, we explored RAG, AI embeddings, vector databases, and semantic search.

Now we come to one of the most common questions when building AI applications:

Should I use RAG or Fine-Tuning?

Both approaches can customize an AI application, but they solve different problems.

RAG (Retrieval-Augmented Generation) gives an AI model relevant information at query time, while fine-tuning changes the model's behavior by training it further on examples.

In this beginner-friendly guide, we will learn what RAG is, what fine-tuning is, how they differ, when to use each, their advantages and limitations, and when combining both approaches makes sense.

What Is RAG?

RAG stands for Retrieval-Augmented Generation.

RAG allows an LLM to retrieve relevant information from an external knowledge source before generating an answer.

User Question
      ↓
Embedding
      ↓
Search Knowledge Base
      ↓
Relevant Documents
      ↓
LLM
      ↓
Answer

The important idea is that the information does not have to be permanently stored inside the model's parameters.

Simple RAG Example

Imagine your company has an internal HR document:

Employees receive 20 days
of paid annual leave.

A user asks:

"How many annual leave days do I get?"

The RAG system searches the company knowledge base and retrieves the relevant document.

Question
   ↓
Semantic Search
   ↓
Leave Policy
   ↓
LLM
   ↓
"Employees receive 20 days
of paid annual leave."

The LLM uses the retrieved information to generate the answer.

What Is Fine-Tuning?

Fine-tuning is the process of taking an already trained model and training it further on a specialized dataset.

Pretrained Model
      ↓
Specialized Training Data
      ↓
Fine-Tuning
      ↓
Customized Model

Fine-tuning changes the model's learned parameters so that it becomes better suited to a particular task, style, format, or behavior.

Simple Fine-Tuning Example

Suppose you want a model to consistently respond in a particular format.

Training examples might look like:

User:
Create a support ticket.

Assistant:
{
  "category": "Technical",
  "priority": "High",
  "summary": "..."
}

After training on many high-quality examples, the model can become better at following this desired output pattern.

The goal is not simply to give the model a document to look up. The goal is to improve the model's behavior on a particular task.

RAG vs Fine-Tuning in One Sentence

RAG	Fine-Tuning
Provides additional information to the model at runtime	Further trains the model on examples
Primarily changes what information the model can access	Primarily changes how the model behaves on the trained task
Knowledge can be updated by updating the source data	New learned behavior generally requires another training process

RAG Does Not Train the LLM

This is an important concept.

When you add a company PDF to a RAG system, you are normally not training the LLM on that PDF.

Instead, the document is indexed and retrieved when needed.

Company PDF
     ↓
Extract Text
     ↓
Chunking
     ↓
Embeddings
     ↓
Vector Database
     ↓
Retrieve Relevant Chunk
     ↓
LLM

The underlying model parameters remain unchanged.

Fine-Tuning Changes the Model

Fine-tuning is different.

Base Model
    ↓
Training Examples
    ↓
Fine-Tuning
    ↓
Updated Model
    ↓
New Behavior

The model is further trained so that it can learn patterns represented in the training dataset.

RAG Is Like Giving the AI a Reference Book

A simple way to understand RAG is to imagine an employee taking an exam.

With RAG, the employee can access a reference book while answering the question.

Question
   +
Reference Material
   ↓
Answer

The reference material can be updated without retraining the employee.

Fine-Tuning Is Like Additional Training

Fine-tuning is more like giving the employee additional training.

Existing Knowledge
       +
Specialized Training
       ↓
Improved Task Behavior

The employee learns patterns from the training examples.

When Should You Use RAG?

RAG is usually a good choice when the AI needs access to external, changing, private, or domain-specific information.

Examples include:

• Company policies
• Product documentation
• Technical documentation
• Customer support knowledge bases
• Internal databases
• Legal documents
• Frequently changing business information
• Research documents

Example: Company HR Assistant

Imagine you build an HR chatbot.

The knowledge base contains:

Leave Policy
Insurance Policy
Travel Policy
Remote Work Policy
Employee Handbook

An employee asks:

"Can I carry unused leave to next year?"

RAG can retrieve the relevant policy and provide it to the LLM.

Employee Question
       ↓
Semantic Search
       ↓
Leave Policy
       ↓
LLM
       ↓
Answer

If the company changes its leave policy next month, you can update the knowledge base without retraining the LLM.

When Should You Use Fine-Tuning?

Fine-tuning can be useful when you want to improve a model's performance on a particular task or make its behavior more consistent.

Examples include:

• Specific response formats
• Classification tasks
• Domain-specific language patterns
• Consistent tone or style
• Structured output patterns
• Specialized task behavior

Example: Customer Support Classification

Suppose you want an AI system to classify support tickets.

"My payment was declined."
        ↓
     Payment

"My password doesn't work."
        ↓
     Account

"The application crashes."
        ↓
     Technical

If you have a large, high-quality dataset of representative examples, fine-tuning may help the model become better at the classification task.

RAG vs Fine-Tuning: Knowledge vs Behavior

A useful mental model is:

RAG
 ↓
"What information should the model see?"

Fine-Tuning
 ↓
"How should the model behave?"

This is not an absolute rule, but it is a useful starting point when designing an AI application.

What About Frequently Changing Information?

Suppose your application needs information that changes every day.

Examples:

• Product prices
• Inventory
• Company policies
• News
• Schedules
• Customer records

RAG or another runtime data-access mechanism is generally more suitable than repeatedly fine-tuning the model.

New Information
      ↓
Update Knowledge Source
      ↓
Retrieval
      ↓
LLM
      ↓
Current Answer

What About Changing the AI's Style?

Suppose you want the model to consistently produce responses in a particular format.

Input:
Create a support response.

Desired format:

Title:
Summary:
Resolution:
Next Steps:

Fine-tuning may be useful when you have enough high-quality examples and need consistent task behavior.

However, prompting and structured-output techniques should usually be evaluated first because they can be simpler than fine-tuning.

RAG Does Not Automatically Prevent Hallucinations

RAG can provide the LLM with relevant source information, but it does not guarantee that the generated answer will always be correct.

For example:

Question
   ↓
Retrieve Relevant Document
   ↓
LLM
   ↓
Potentially Incorrect Answer

The retrieved context itself may be incomplete, outdated, ambiguous, or irrelevant.

Good RAG systems therefore need careful retrieval, prompt design, evaluation, source handling, and access control.

Does Fine-Tuning Eliminate Hallucinations?

No.

Fine-tuning does not automatically make a model factually reliable.

A fine-tuned model can still generate incorrect information.

Fine-tuning should therefore not be considered a replacement for reliable data retrieval or application-level validation when factual accuracy is important.

RAG vs Fine-Tuning: Data Requirements

RAG	Fine-Tuning
Requires a useful knowledge source	Requires high-quality training examples
Documents can be updated independently	Training data is incorporated during training
Usually focuses on retrieval quality	Focuses on learning task-specific patterns

RAG vs Fine-Tuning: Updating Information

Consider a company changing its travel policy.

Using RAG

Updated Policy
     ↓
Update Knowledge Base
     ↓
Generate/Update Embeddings
     ↓
Available to RAG
     ↓
LLM

Using Fine-Tuning

Updated Policy
     ↓
Prepare Training Data
     ↓
Fine-Tuning
     ↓
Updated Model
     ↓
Deployment

For frequently changing factual information, maintaining an external knowledge source is often much more practical.

RAG vs Fine-Tuning: Cost

The cost depends heavily on the architecture, model, dataset size, infrastructure, and usage pattern.

In general, RAG requires investment in components such as:

• Document processing
• Embedding generation
• Vector storage
• Retrieval infrastructure

Fine-tuning requires resources for:

• Preparing training data
• Training or fine-tuning infrastructure
• Evaluation
• Model storage
• Deployment

The cheapest approach depends on the specific application, so it is better to compare the total system cost rather than assuming one approach is always cheaper.

RAG vs Fine-Tuning: Latency

RAG introduces a retrieval step before generation.

Question
 ↓
Embedding
 ↓
Search
 ↓
Retrieve Context
 ↓
LLM
 ↓
Answer

This can add latency compared with sending a prompt directly to a model.

Fine-tuning does not require document retrieval for every query, although the overall application can still use other external data sources.

RAG vs Fine-Tuning: Privacy

Privacy requirements depend on the deployment architecture and provider.

For enterprise applications, you should consider:

• Where documents are stored
• Where embeddings are generated
• Where inference occurs
• Who can access the data
• How data is encrypted
• Data retention policies
• Tenant isolation

Neither RAG nor fine-tuning is automatically private or insecure. Security depends on how the complete system is designed and operated.

Can RAG and Fine-Tuning Be Used Together?

Yes.

RAG and fine-tuning are not mutually exclusive.

A system can use a fine-tuned model together with a RAG pipeline.

User Question
      ↓
Semantic Search
      ↓
Relevant Documents
      ↓
Fine-Tuned LLM
      ↓
Answer

For example, the model could be fine-tuned to follow a company's response format while RAG provides the latest company information.

Example: Enterprise AI Assistant

Imagine a company wants an AI assistant that answers questions about internal systems.

Requirements:

• Use internal documentation
• Follow a consistent response format
• Use current information
• Provide relevant sources

A possible architecture could be:

Company Documents
       ↓
Embeddings
       ↓
Vector Database
       ↓
Semantic Search
       ↓
Relevant Context
       ↓
Fine-Tuned LLM
       ↓
Structured Answer

Here:

• RAG provides the relevant information.
• Fine-tuning can help with specialized behavior or formatting.

Should You Fine-Tune First?

Usually, you should not start with fine-tuning simply because your application needs company-specific knowledge.

First determine whether the problem can be solved with:

• Good prompting
• Structured outputs
• RAG
• Tool calling
• Better retrieval

Fine-tuning becomes more attractive when you have a clear task-specific behavior that is difficult to achieve reliably with prompting and other simpler techniques.

A Simple Decision Guide

Need current or private documents?
             ↓
            RAG

Need the model to learn a specific behavior?
             ↓
        Fine-Tuning

Need both current knowledge
and specialized behavior?
             ↓
       RAG + Fine-Tuning

RAG vs Fine-Tuning Comparison

Feature	RAG	Fine-Tuning
Main purpose	Provide relevant external information	Adapt model behavior to a task
Changes model parameters	No	Yes
Best for changing knowledge	Yes	Usually not
Best for task-specific behavior	Sometimes	Yes
Uses external knowledge at runtime	Yes	Not inherently
Requires vector search	Commonly	No
Requires training examples	Not necessarily	Yes
Easy to update factual knowledge	Yes	No

Common Misconceptions

1. RAG Trains the LLM

False. RAG normally retrieves information and puts it into the model's context. It does not update the model's parameters.

2. Fine-Tuning Is the Best Way to Add Documents

Usually false. If the goal is to give the model access to frequently changing documents, RAG is often more suitable.

3. Fine-Tuning Makes the Model Know Everything

False. Fine-tuning teaches patterns from training examples. It is not a general-purpose replacement for a knowledge retrieval system.

4. RAG and Fine-Tuning Are Competitors

Not necessarily. They can complement each other in the same application.

Real-World Example

Suppose you build an AI assistant for a software company.

You want the assistant to:

• Answer questions using current documentation.
• Follow a consistent support format.
• Understand company-specific terminology.

A possible solution is:

Company Documentation
        ↓
     Chunking
        ↓
    Embeddings
        ↓
  Vector Database
        ↓
  Semantic Search
        ↓
 Relevant Context
        ↓
Fine-Tuned / Instruction-Following LLM
        ↓
      Answer

RAG provides the knowledge, while model customization can help provide the desired behavior.

Key Takeaways

• RAG stands for Retrieval-Augmented Generation.
• Fine-tuning means further training a pretrained model on specialized examples.
• RAG commonly helps an LLM access external and changing information.
• Fine-tuning can help a model learn specialized task behavior.
• RAG does not normally change the model's parameters.
• Fine-tuning changes model parameters during training.
• RAG is often useful for private and frequently changing knowledge.
• Fine-tuning can be useful for consistent task-specific behavior.
• Prompting and structured outputs should often be evaluated before fine-tuning.
• RAG and fine-tuning can be used together.

Conclusion

RAG and fine-tuning solve different problems.

If you need an AI application to access current, private, or frequently changing information, RAG is often the better starting point.

If you need the model to perform a specific task more consistently or follow specialized patterns, fine-tuning may be appropriate.

And in some advanced applications, you can combine both:

RAG
 ↓
Current Knowledge
 +
Fine-Tuning
 ↓
Specialized Behavior
 +
LLM
 ↓
Better AI Application

Next: Now that we understand RAG, embeddings, vector databases, semantic search, and fine-tuning, the next step is to build something practical: Build a RAG Application with C# and .NET.