In the previous articles, we learned what RAG (Retrieval-Augmented Generation) is and how a RAG system uses chunking, vector databases, semantic search, and LLMs.
One of the most important technologies behind this process is AI embeddings.
Embeddings allow AI systems to represent text as numbers so that computers can compare the meaning of different pieces of information.
In this beginner-friendly guide, we will learn what embeddings are, how they work, what vectors and dimensions mean, how similarity is calculated, and why embeddings are important for RAG and semantic search.
What Are AI Embeddings?
An embedding is a numerical representation of data that captures useful characteristics or semantic relationships.
For text, an embedding model converts a sentence, paragraph, or document into a vector of numbers.
Text ↓ Embedding Model ↓ Vector ↓ [0.12, -0.45, 0.78, 0.21, ...]
The resulting vector can contain hundreds or thousands of numerical values depending on the embedding model.
You can think of an embedding as a way of converting human-readable information into a mathematical representation that an AI system can compare.
Simple Example
Consider these two sentences:
Sentence A: How many vacation days do I have? Sentence B: Employees receive 20 days of annual leave.
The words are not exactly the same, but the meanings are closely related.
An embedding model can convert both sentences into vectors:
Sentence A
↓
Vector A
Sentence B
↓
Vector B
The vectors can then be compared to determine how semantically similar the sentences are.
Why Do AI Systems Need Embeddings?
Computers are very good at processing numbers, but human language is complex.
Consider these questions:
"How many vacation days do I get?" "How much annual leave am I entitled to?" "What's my yearly leave allowance?"
A keyword search may treat these as different queries because the words are different.
However, their meanings are very similar.
Embeddings help represent these meanings mathematically so that an AI system can identify the relationship.
Text to Vector
The basic process is:
Human Text
↓
Embedding Model
↓
Numerical Vector
For example:
"I love programming"
↓
[0.034, -0.182, 0.731, 0.245, -0.093, ...]
The numbers themselves are not normally meaningful to a human. Their usefulness comes from how vectors relate to each other in the embedding space.
What is a Vector?
A vector is simply an ordered collection of numbers.
[0.25, -0.13, 0.87, 0.42]
In AI, vectors can represent many types of information, including:
- Text
- Images
- Audio
- Documents
- User preferences
- Products
In RAG applications, vectors are most commonly used to represent document chunks and user questions.
What Are Dimensions?
The number of values in an embedding vector is called its dimension or dimensionality.
For example:
[0.12, 0.45, -0.31]
This vector has 3 dimensions.
A real embedding model might produce a much larger vector:
[0.12, 0.45, -0.31, 0.72, ... many more values ...]
The exact dimensionality depends on the embedding model.
What Does an Embedding Represent?
An embedding does not normally assign one obvious human-readable meaning to each individual number.
Instead, the vector as a whole represents information learned by the embedding model.
Think of it like a location on a very large mathematical map.
Similar Meaning
↑
│
A ● │ ● B
│
│
│
│
│
│
└────────────────→
Text with similar meanings tends to be represented closer together in the embedding space.
Embeddings Create a Semantic Space
Imagine a huge mathematical space containing millions of points.
Each point represents an embedding.
AI Embedding Space
Dogs ●
● Puppies
● Cars
● Vehicles
● Cats
● Computers
Related concepts tend to form regions or clusters in the embedding space.
This allows AI applications to perform semantic searches.
Semantic Similarity
Semantic similarity measures how closely two pieces of text are related in meaning.
For example:
"How do I reset my password?" "What should I do if I forgot my password?"
These sentences use different words but express a similar intent.
Their embeddings can therefore be relatively close to each other.
Keyword Search vs Semantic Search
Keyword Search
Traditional keyword search looks for matching terms.
Query: "vacation days" Document: "Employees receive 20 days of annual leave."
There may be no exact match for the word vacation.
Semantic Search
Semantic search uses embeddings to compare meaning.
Query:
"How many vacation days do I get?"
↓
Embedding
↓
Vector Search
↓
Document:
"Employees receive 20 days
of annual leave."
The system can recognize that vacation days and annual leave are semantically related.
How Are Embeddings Used in RAG?
Embeddings are a fundamental part of a typical RAG architecture.
Documents
↓
Chunking
↓
Embedding Model
↓
Vectors
↓
Vector Database
When the user asks a question:
User Question
↓
Embedding Model
↓
Query Vector
↓
Vector Database
↓
Similar Document Vectors
↓
Relevant Chunks
↓
LLM
↓
Answer
Document Embeddings
During the indexing process, each document chunk is converted into an embedding.
Document Chunk 1
↓
Embedding 1
Document Chunk 2
↓
Embedding 2
Document Chunk 3
↓
Embedding 3
These embeddings are then stored in a vector database.
Query Embeddings
When a user asks a question, the question is converted into an embedding using the same or compatible embedding model used for the indexed content.
User Question
↓
Embedding Model
↓
Query Vector
The query vector is then compared with the stored document vectors.
How Does Vector Similarity Work?
Once we have two vectors, we need a way to measure how similar they are.
Common approaches include:
- Cosine Similarity
- Dot Product
- Euclidean Distance
The exact choice depends on the embedding model and vector database.
Cosine Similarity
Cosine similarity measures the angle between two vectors.
Conceptually:
Vector A
↘
↘
↘
↘ Vector B
Small angle
↓
High similarity
When using the common cosine-similarity convention, a value closer to 1 generally indicates greater similarity, while a value closer to 0 indicates weaker similarity.
Negative values can also occur depending on the vectors and model.
Simple Similarity Example
Imagine we have three documents:
Document A: How to reset your password Document B: Password recovery instructions Document C: How to configure a database
User asks:
"I forgot my password. How can I recover it?"
The query embedding might produce similarity scores such as:
Document A → 0.91 Document B → 0.88 Document C → 0.24
The system would consider Documents A and B much more relevant than Document C.
What is an Embedding Model?
An embedding model is a machine learning model specifically designed to convert input data into vector representations.
For text:
Text ↓ Embedding Model ↓ Vector
Different embedding models have different characteristics, such as:
- Embedding dimension
- Supported languages
- Context length
- Semantic quality
- Speed
- Memory requirements
Embedding Model vs LLM
An embedding model and an LLM perform different jobs.
| Embedding Model | LLM |
|---|---|
| Converts content into vectors | Generates text |
| Used for similarity and retrieval | Used for reasoning and generation |
| Produces numerical representations | Produces natural-language responses |
| Commonly used before retrieval | Commonly used after retrieval in RAG |
A RAG system may therefore use both an embedding model and an LLM.
Why Not Use the LLM to Search Documents?
An LLM is designed primarily for understanding and generating language. A vector search system provides an efficient way to find relevant information across a large collection of embedded content.
A typical architecture separates these responsibilities:
Embedding Model
↓
Find Relevant Information
↓
Vector Database
↓
Retrieve Context
↓
LLM
↓
Generate Answer
This separation makes it possible to search a large knowledge base without sending every document to the LLM.
Embeddings and Vector Databases
The relationship between embeddings and vector databases is important.
Embedding Model
↓
Creates Vectors
↓
Vector Database
↓
Stores & Searches Vectors
The embedding model creates the representation, while the vector database provides infrastructure for storing and retrieving those representations.
Example Vector Database Record
A RAG system might store information similar to:
ID: DOC-001-CHUNK-05 Text: Employees receive 20 days of annual leave. Embedding: [0.12, -0.42, 0.71, ...] Metadata: Document = LeavePolicy.pdf Page = 12 Department = HR Year = 2026
The metadata can later be used for filtering and displaying source information.
Embeddings for Images
Embeddings are not limited to text.
Images can also be represented as vectors.
Image ↓ Embedding Model ↓ Image Vector
This makes it possible to build systems that search for images based on visual or semantic similarity.
For example:
Query:
"red sports car"
↓
Embedding
↓
Search Image Vectors
↓
Similar Images
Multimodal Embeddings
Some modern AI systems can represent different types of content in compatible embedding spaces.
This can enable relationships between:
- Text and images
- Images and text
- Audio and text
- Other forms of multimodal data
This area is especially useful for applications that need to search across multiple types of content.
What Happens When a Document Changes?
Suppose your company updates its leave policy.
The old document can be replaced or re-indexed.
Updated Document
↓
Extract Text
↓
Chunk
↓
Generate New Embeddings
↓
Update Vector Database
↓
New Information Available
The underlying LLM does not need to be retrained simply because the document changed.
Embedding Quality Matters
The quality of a RAG system depends heavily on how well the embedding model represents your data.
A poor embedding model may produce weak retrieval results even when the rest of the system is well designed.
When selecting an embedding model, consider:
- Language support
- Domain suitability
- Retrieval quality
- Latency
- Infrastructure requirements
- Cost
- Maximum input length
Embedding Model and Language
If your application contains multiple languages, the embedding model's language support becomes important.
For example, a multilingual application may contain:
English Tamil Hindi Japanese German
A multilingual embedding model may be more appropriate than a model optimized only for English.
Embeddings in a Real RAG Application
Let's look at the complete process using a company knowledge base.
Suppose we have:
Leave Policy.pdf Travel Policy.pdf Insurance Policy.pdf IT Security.pdf
During indexing:
PDF ↓ Text Extraction ↓ Chunks ↓ Embedding Model ↓ Vectors ↓ Vector Database
Later, a user asks:
"Can I carry unused vacation days into next year?"
The question becomes an embedding:
Question ↓ Embedding Model ↓ Query Vector
The vector database searches for similar vectors and may return a chunk from the leave policy.
Query Vector
↓
Vector Search
↓
Leave Policy Chunk
↓
LLM
↓
Final Answer
Embeddings and RAG Performance
Several factors can affect retrieval performance:
- Embedding model quality
- Chunking strategy
- Chunk size
- Chunk overlap
- Similarity metric
- Top-K value
- Metadata filtering
- Reranking
- Document quality
This means that improving a RAG system is not simply a matter of selecting a larger LLM.
Embeddings vs Keywords: A Simple Comparison
| Keyword Search | Embedding Search |
|---|---|
| Matches words | Compares semantic representations |
| Good for exact terms | Good for semantic relationships |
| Simple and efficient | Requires embedding generation |
| Useful for IDs and exact names | Useful for natural-language questions |
Many modern applications use hybrid search to combine both approaches.
Common Mistakes with Embeddings
1. Using Different Embedding Spaces
Document embeddings and query embeddings generally need to be generated using compatible models and configurations.
2. Ignoring Chunking
Even a strong embedding model cannot fix poorly structured chunks.
3. Retrieving Too Many Results
More results do not necessarily mean a better answer. Irrelevant context can reduce answer quality.
4. Ignoring Metadata
Metadata can help restrict searches and improve relevance.
5. Assuming Similarity Means Correctness
A high similarity score means that content is considered similar according to the retrieval system. It does not automatically mean that the retrieved information is factually correct.
Embeddings in One Simple Diagram
DOCUMENT
↓
Chunking
↓
Embedding Model
↓
Vector
↓
Vector Database
│
│
↓
Similarity Search
↑
│
User Query
↓
Embedding Model
↓
Query Vector
│
↓
Relevant Documents
↓
LLM
↓
Answer
Key Takeaways
- Embedding: A numerical representation of data.
- Vector: An ordered collection of numbers.
- Dimension: The number of values in an embedding.
- Embedding Model: A model that converts data into vectors.
- Semantic Search: Search based on meaning rather than only exact words.
- Vector Database: Stores and searches vector representations.
- Similarity: Measures how closely two vectors are related.
- RAG: Uses embeddings to retrieve relevant information before generating an answer.
Conclusion
AI embeddings are one of the fundamental building blocks of modern AI applications.
They provide a way to represent text and other data as vectors, allowing applications to compare information based on similarity and meaning.
In a RAG system, embeddings connect the user's question with the knowledge stored in documents:
Text ↓ Embedding ↓ Vector ↓ Similarity Search ↓ Relevant Information ↓ LLM ↓ Answer
Once you understand embeddings, the next important concept is the technology that stores and searches these vectors: the Vector Database.
