What Is RAG?
Retrieval-Augmented Generation, commonly called RAG, is an architecture pattern that allows an AI application to retrieve relevant information from an organization's data before asking a large language model to generate a response.
Instead of relying only on information learned during model training, the application can retrieve current, organization-specific information and provide it to the model as context.
The RAG Architecture
Why Organizations Use RAG
Enterprise information changes continuously. Policies are updated, documents are replaced, procedures change, and new information becomes available.
A RAG architecture can retrieve current information from governed enterprise sources instead of requiring the language model itself to contain all of that information.
- Connect AI applications to enterprise information.
- Retrieve information that is relevant to the question.
- Use current or frequently updated source information.
- Preserve relationships to source documents.
- Apply metadata and authorization controls.
- Provide supporting context to the language model.
Step 1: Prepare the Enterprise Data
RAG begins long before a user asks a question. The source information must first be prepared for retrieval.
Step 2: Convert the User Question into a Searchable Representation
When a user asks a question, the application can transform the question into an embedding using an embedding model.
The resulting vector can then be compared with vectors representing stored document chunks.
Step 3: Apply Metadata Filters
Vector similarity alone is not enough for enterprise retrieval.
The retrieval process can use metadata to narrow the candidate information before it becomes context for the language model.
Example metadata might include:
- Document type
- Department
- Business unit
- Effective date
- Document status
- Security classification
- Geographic region
- Source system
Step 4: Authorization
After relevant information is identified, the application must determine whether the requesting user is authorized to access that information.
Relevance ≠ Authorization
A document can be highly relevant to a user's question and still be unauthorized for that user.
This distinction is one of the most important governance principles in an enterprise RAG architecture.
Step 5: Build the Authorized Context
Once the system has identified information that is both relevant and authorized, the selected chunks can be assembled into context for the language model.
The context may contain several retrieved chunks from one or more documents.
"How many vacation days can an employee carry over?"
The RAG system retrieves the authorized sections of the current Vacation Policy and provides those sections to the LLM as context.
Step 6: Send Context to the LLM
The language model receives the user's question together with the retrieved context.
The model can then generate a response based on the supplied information.
Step 7: Preserve Lineage
An enterprise RAG system should be able to trace retrieved information back to its original source.
AI ANSWER
│
▼
RETRIEVED CHUNK
│
▼
CHK-0003
│
▼
DOC-10001
│
▼
Vacation_Policy.pdf
│
▼
HR SharePoint
This creates a traceable relationship between the AI response and the source information used to generate it.
The RAG Data Model
From a data architecture perspective, RAG introduces relationships across source data, chunks, embeddings, vectors, retrieval events, and AI responses.
DOCUMENT
│
│ 1 : N
▼
DOCUMENT_CHUNK
│
│ 1 : 1
▼
EMBEDDING
│
│ 1 : 1
▼
VECTOR
│
│
▼
VECTOR SEARCH
│
▼
RETRIEVED CHUNK
│
▼
AUTHORIZED CONTEXT
│
▼
LLM
│
▼
AI RESPONSE
Governance Controls in RAG
RAG introduces several governance requirements that extend traditional data governance into the AI layer.
- Source ownership
- Data quality
- Metadata management
- Document version control
- Source lineage
- Security classification
- Authorization
- Retrieval controls
- Embedding model governance
- Vector lifecycle management
- Audit logging
- Monitoring and validation
What Happens When a Source Document Changes?
Consider a company vacation policy that changes from Version 3 to Version 4.
The organization must have a controlled process for determining what happens to vectors associated with the previous version.
What Happens When Authorization Is Revoked?
Another important governance scenario occurs when a source document is no longer authorized for AI use.
Example Control
Authorization revoked
↓
Block or deactivate the associated document
↓
Prevent retrieval and AI use
↓
Create audit event
↓
Archive according to policy
Complete RAG Architecture
The US-IDMS Perspective
RAG should not be treated simply as an LLM integration pattern.
From an enterprise architecture perspective, RAG is a governed data pipeline that connects source information to an AI application.
The US-IDMS methodology is:
- Assess the source data and information environment.
- Architect the ingestion, retrieval, and AI data architecture.
- Govern metadata, lineage, security, authorization, and lifecycle.
- Implement the retrieval and AI solution.
- Control access, retrieval, source usage, and AI behavior.
- Monitor quality, performance, lineage, and governance.
The quality of the AI response ultimately depends on the quality, relevance, lineage, security, authorization, and governance of the information supplied to the model.
Building Trusted AI Data Foundations
US-IDMS helps organizations assess their data environment, design AI-ready architectures, establish governance, implement solutions, and put controls around AI data and retrieval.
Discuss Your Data Architecture