What Is AI Sovereignty? The New Buzzword Explained?

Image
  AI EXPLAINED  POLICY GUIDE  AI SOVEREIGNTY  2026 What Is AI Sovereignty? The New Buzzword Explained Why Every G20 Government Suddenly Has an AI Independence Strategy — and Why Most of Them Can't Fully Deliver It Search volume for "sovereign AI" tripled between January 2025 and April 2026. McKinsey projects the market at $500–600 billion by 2030. And yet the Boston Consulting Group's own verdict is blunt: for most countries, AI sovereignty as full-stack independence is an illusion. Here's what the term actually means, and what's genuinely achievable. AI sovereignty has gone from a niche policy phrase to a line item in most G20 national budgets in roughly two years. France has Mistral, the UAE has G42 and Falcon, Saudi Arabia has HUMAIN, Singapore has SEA-LION, India has BharatGen, Japan has LLM-jp, and the UK launched a £500 million Sovereign AI Fund in April 2026 under an explicit slogan of becoming "an AI maker, not an AI taker." What was ...

How to Build a RAG App With Lang Chain

 AI TOOLS  DEVELOPER GUIDE  LANGCHAIN  2026

Editorial illustration of documents flowing through a pipeline into a glowing database and out to a chat bubble, representing a RAG pipeline

How to Build a RAG App With LangChain

A Complete, Working Walkthrough — From a Blank File to a Retrieval Pipeline That Actually Holds Up

LangChain's own official RAG tutorial gets a working app running in about 40 lines of Python — a genuinely good way to learn the shape of the problem, and a genuinely bad production design. This guide builds the same core pipeline, then walks through exactly what changes once real documents, real users, and real follow-up questions show up.

Retrieval-Augmented Generation, or RAG, solves a specific, common problem: a large language model only knows what it was trained on, and re-training a model every time your documents change isn't remotely practical. RAG instead fetches relevant chunks of your own data at the moment someone asks a question, and hands those chunks to the model as context — so the model answers using your actual documents, not just what it happened to learn during training.

LangChain has become the most widely used framework for wiring this together, with well over 100,000 GitHub stars and millions of monthly downloads across its Python and JavaScript ecosystems. It handles the unglamorous plumbing — loading documents, splitting them into chunks, calling an embedding model, talking to a vector database, and composing all of it with a language model — through a consistent set of interfaces that work whether you're using Claude, GPT, Gemini, or a fully local open-source model.

This guide builds a working RAG application step by step using LangChain's current v1.x interface with LangChain Expression Language (LCEL), then covers the specific changes that separate a working prototype from something you'd actually trust in production — chunking strategy, conversational memory, hybrid search, and evaluation. Every code block is complete and ordered so you can build the whole thing by following along top to bottom.

One framing note before diving in: this guide uses Claude via langchain-anthropic for generation, since that's the model family this site covers in depth elsewhere, but every pattern shown here is provider-agnostic by design. LangChain's standard chat model interface means swapping ChatAnthropic for ChatOpenAI or a Gemini equivalent is a one-line change — the retrieval logic, prompt structure, and chain composition around it stay identical regardless of which model is actually generating the final answer.

ALSO READ THIS: AI + Cybersecurity: How Hackers Are Using AI Now

 

In Two Sentences

A RAG app built with LangChain loads your documents, splits them into chunks, embeds those chunks into a vector store, and — at query time — retrieves the most relevant chunks and hands them to a language model alongside the user's question, so answers stay grounded in your actual data instead of the model's training knowledge alone. The basic pipeline is genuinely simple to build; the real engineering work is in chunking strategy, retrieval quality, and evaluation — the parts a 40-line tutorial skips entirely.

 

100K+

GitHub stars on LangChain, making it the most widely adopted LLM application framework

~$0.01

Typical cost of a single RAG query with 4 retrieved chunks and a 500-token response

4

Core pipeline stages every RAG app needs: load, split, embed and store, retrieve and generate

 

1. What You'll Need Before Starting

This tutorial assumes basic Python familiarity and a terminal you're comfortable working in. You'll need Python 3.9 or later, an API key for whichever LLM provider you choose (this guide uses Claude via langchain-anthropic, though the pattern is identical for OpenAI or Gemini), and — for the embedding step — either an OpenAI API key or a local embedding model if you'd rather avoid a second provider.

Start by creating an isolated project directory and virtual environment, which keeps this project's dependencies separate from anything else on your system.

Terminal

mkdir langchain-rag-app && cd langchain-rag-app

python3 -m venv venv

source venv/bin/activate   # Windows: venv\Scripts\activate

 

pip install langchain langchain-anthropic langchain-openai \

            langchain-community langchain-chroma \

            chromadb pypdf

A quick note on package names, since this trips up a lot of people coming back to LangChain after a break: as of v0.3 and the current v1.x line, LangChain split its provider integrations into separate packages — langchain-anthropic for Claude, langchain-openai for OpenAI, langchain-community for open-source models via Ollama and Hugging Face — rather than bundling every provider into the core library. Installing only what you need keeps your dependency tree smaller and your install times faster.

2. Step 1 — Load and Split Your Documents

The first stage of any RAG pipeline is getting your source documents into memory and breaking them into chunks small enough to embed and retrieve individually. LangChain's document loaders handle dozens of formats — PDF, Markdown, HTML, plain text, even entire directories — through a consistent interface.

ingest.py

from langchain_community.document_loaders import DirectoryLoader, PyPDFLoader

from langchain_text_splitters import RecursiveCharacterTextSplitter

 

# Load every PDF in a folder

loader = DirectoryLoader(

    "./documents",

    glob="**/*.pdf",

    loader_cls=PyPDFLoader,

)

raw_documents = loader.load()

 

# Split into overlapping chunks that preserve context

splitter = RecursiveCharacterTextSplitter(

    chunk_size=1000,

    chunk_overlap=200,

    separators=["\n\n", "\n", ". ", " ", ""],

)

chunks = splitter.split_documents(raw_documents)

 

print(f"Loaded {len(raw_documents)} documents")

print(f"Split into {len(chunks)} chunks")

The chunk_size and chunk_overlap values here — 1,000 characters with a 200-character overlap — are a reasonable starting default, not a universal answer. The overlap exists specifically so a sentence or idea that gets cut off at a chunk boundary still appears in full inside at least one chunk, rather than being split across two chunks in a way that damages its meaning. Section 6 covers how and why to tune these numbers for your actual documents.

It's worth understanding why RecursiveCharacterTextSplitter specifically is the right default over a simpler fixed-length splitter. Rather than cutting text at exactly character 1,000 regardless of what's there, it works through its list of separators in order of preference — trying to split on paragraph breaks first, then line breaks, then sentence endings, and only falling back to splitting mid-word as a last resort. The practical effect is that chunks tend to end at natural language boundaries far more often than a naive fixed-length approach would produce, which measurably helps both embedding quality and how coherent a retrieved chunk reads once it's handed to the language model as context.

3. Step 2 — Create Embeddings and Store Them

Each chunk needs to be converted into an embedding — a vector of numbers that captures its meaning — so the system can later find chunks that are semantically similar to a user's question, not just chunks that share exact keywords. LangChain's Chroma integration handles both the embedding call and the storage in a single step for a local, file-based vector store, which is the right choice for a prototype or a small-to-medium production workload.

ingest.py (continued)

from langchain_openai import OpenAIEmbeddings

from langchain_chroma import Chroma

 

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

 

vector_store = Chroma.from_documents(

    documents=chunks,

    embedding=embeddings,

    persist_directory="./chroma_db",

)

 

print("Vector store built and saved to ./chroma_db")

text-embedding-3-small is a reasonable default for most projects in 2026 — it offers a strong balance of retrieval quality, speed, and cost, at roughly two cents per million tokens embedded. If you're working with dense technical, legal, or medical documents where retrieval precision matters more than cost, it's worth benchmarking a higher-end embedding model, such as one from Voyage AI or Cohere, against your own document set before committing — the right choice genuinely depends on what you're embedding, and the cost difference at typical document volumes is usually small relative to what a meaningful accuracy improvement is worth.

ALSO READ THIS: What Is Generative Engine Optimization (GEO)?

4. Step 3 — Build the Retriever and RAG Chain

With your documents embedded and stored, the query-time pipeline is where LangChain Expression Language (LCEL) does its real work — composing the retriever, prompt template, and language model into a single callable chain using Python's pipe operator, which makes the data flow through each stage easy to read top to bottom.

query.py

from langchain_anthropic import ChatAnthropic

from langchain_openai import OpenAIEmbeddings

from langchain_chroma import Chroma

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.output_parsers import StrOutputParser

from langchain_core.runnables import RunnablePassthrough

 

# Reconnect to the vector store built during ingestion

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

vector_store = Chroma(

    persist_directory="./chroma_db",

    embedding_function=embeddings,

)

retriever = vector_store.as_retriever(search_kwargs={"k": 4})

 

# Prompt template — instructs the model to answer only from context

prompt = ChatPromptTemplate.from_template("""

Answer the question using only the context below.

If the context doesn't contain the answer, say you don't know.

 

Context:

{context}

 

Question: {question}

""")

 

llm = ChatAnthropic(model="claude-sonnet-4-6", temperature=0)

 

def format_docs(docs):

    return "\n\n".join(doc.page_content for doc in docs)

 

rag_chain = (

    {"context": retriever | format_docs, "question": RunnablePassthrough()}

    | prompt

    | llm

    | StrOutputParser()

)

 

answer = rag_chain.invoke("What does the onboarding policy say about remote work?")

print(answer)

Reading this chain from the inside out: RunnablePassthrough() lets the user's raw question flow through unchanged into the question slot of the prompt, while the retriever pulls the four most relevant chunks (k=4) and format_docs joins them into a single block of text for the context slot. The pipe operator chains each stage's output directly into the next stage's input — retriever results into the prompt, the filled prompt into the model, and the model's response through an output parser that returns a clean string instead of a raw message object.

The instruction to answer only from the provided context, and to say so plainly when the context doesn't contain an answer, is doing real work here — it's the single most effective prompt-level lever for keeping a RAG app from confidently making something up when the retrieved chunks simply don't cover what was asked.

Simple educational illustration showing document, split, store, and chat icons connected by arrows, representing the RAG pipeline stages

5. Step 4 — Add Conversational Memory

The chain above works well for a single, standalone question, but it breaks down the moment a user asks a natural follow-up. If someone first asks "What's the deployment process?" and then follows up with "How long does it take?", the retriever has no way to know that "it" refers to the deployment process — it will search for documents about the word "it," which is meaningless. The fix is to reformulate follow-up questions using conversation history before retrieval ever happens.

query.py (conversational version)

from langchain.chains import create_history_aware_retriever, create_retrieval_chain

from langchain.chains.combine_documents import create_stuff_documents_chain

from langchain_core.prompts import MessagesPlaceholder

 

# Prompt that rewrites a follow-up question into a standalone one

condense_prompt = ChatPromptTemplate.from_messages([

    MessagesPlaceholder("chat_history"),

    ("user", "{input}"),

    ("user", "Given the conversation above, rewrite the last "

              "question as a standalone question that includes "

              "any needed context."),

])

 

history_aware_retriever = create_history_aware_retriever(

    llm, retriever, condense_prompt

)

 

answer_prompt = ChatPromptTemplate.from_messages([

    ("system", "Answer using only this context:\n\n{context}"),

    MessagesPlaceholder("chat_history"),

    ("user", "{input}"),

])

 

document_chain = create_stuff_documents_chain(llm, answer_prompt)

conversational_rag_chain = create_retrieval_chain(

    history_aware_retriever, document_chain

)

 

# Usage — chat_history is a list of prior HumanMessage/AIMessage turns

result = conversational_rag_chain.invoke({

    "input": "How long does it take?",

    "chat_history": chat_history,   # built up turn by turn

})

print(result["answer"])

This two-stage structure — rewrite, then retrieve and answer — is the standard LangChain pattern for conversational RAG as of the current v1.x release line, and it's a meaningfully different shape than simply appending chat history onto a single prompt, which tends to confuse the retriever the same way the raw follow-up question would.

ALSO READ THIS: How to Detect AI-Generated Text — Best Tools

6. Why the Basic Tutorial Isn't a Production Design

Everything above will run and produce reasonable answers on a modest document set. It will also, per widely shared 2026 practitioner writing on exactly this gap, start breaking down in specific, predictable ways once real usage begins — and it's worth naming those gaps directly rather than discovering them after launch.

•      Chunk size is rarely one-size-fits-all: a fixed 1,000-character chunk works reasonably for prose but fragments tables, code blocks, and structured data badly — production pipelines increasingly use format-aware splitting that respects a document's actual structure rather than cutting at a fixed character count.

•      Pure vector similarity search misses exact matches: semantic search is excellent at finding conceptually related content but can miss a chunk containing an exact product code, error message, or proper noun that a simple keyword search would catch instantly — production systems increasingly combine vector search with keyword-based (BM25) search in a hybrid retriever.

•      k=4 is a guess, not a tuned parameter: retrieving too few chunks starves the model of context it needs; retrieving too many dilutes the prompt with irrelevant material and increases cost — the right value depends on your chunk size and document density, and is worth testing rather than assuming.

•      There's no evaluation loop: the code above has no way to tell you whether retrieval quality is actually good, or whether answers are grounded in the retrieved context versus quietly drifting — LangSmith, LangChain's companion observability platform, is built specifically to trace and evaluate exactly this.

It's worth being specific about how these gaps tend to surface in practice, since they rarely announce themselves clearly. A chunking problem usually shows up as answers that are technically correct but oddly incomplete — the model genuinely didn't have the missing information in its context window, not because retrieval failed outright, but because the relevant fact was split across a chunk boundary into two pieces, neither of which scored highly enough to make the top-k cut. A retrieval-quality problem tends to look different: confident, fluent answers that are simply wrong, because the retriever returned chunks that were topically related but didn't actually contain the specific fact being asked about, and the language model — instructed to answer from context — did its best with imperfect material rather than recognizing the gap. Distinguishing between these two failure modes is exactly the kind of diagnosis an evaluation loop and tracing tool make tractable, and exactly what's impossible to do reliably by just reading a handful of outputs and guessing.

Why This Matters Practically

None of these gaps mean the basic pipeline is wrong to start with — it's the fastest way to validate that RAG is even the right approach for your use case before investing in the harder problems. The mistake is treating the 40-line version as a finished product rather than a first draft. Every serious production RAG deployment covered in current LangChain guidance layers hybrid search, tuned chunking, and evaluation on top of exactly the pipeline built in Sections 2 through 5 — it's an extension of the same foundation, not a different architecture.

7. Choosing a Vector Store — Prototype vs. Production

Chroma, used throughout this guide, is a strong default for getting started: it runs locally with no separate infrastructure to manage, persists to disk, and integrates directly with LangChain's retriever interface. It's genuinely fine for production too, up to a meaningful scale — but it's worth knowing what you'd reach for next as data volume or query load grows.

•      FAISS: a fast, in-memory library from Meta, well-suited to prototypes and research work where persistence and multi-user access aren't primary concerns.

•      Pinecone or Weaviate: fully managed, cloud-hosted vector databases built for production scale, handling replication, high query volume, and metadata filtering without you managing the underlying infrastructure yourself.

•      SingleStore or Postgres with pgvector: a strong choice when you want vector search living alongside your existing relational data rather than as a separate specialized system, simplifying your overall data architecture at the cost of some specialized vector-search performance.

LangChain's provider-neutral design means switching between any of these later is a matter of changing the integration class and its configuration, not rewriting your pipeline logic — the retriever interface built in Section 4 stays identical regardless of which vector store sits behind it.

8. What This Looks Like in Practice

📄 An Internal Policy Chatbot

The exact pipeline from Sections 2–5, pointed at HR documents

A company loads its employee handbook, benefits documentation, and IT policies through the ingestion pipeline in Section 2, then deploys the conversational chain from Section 5 behind a simple internal chat interface, letting employees ask natural-language questions about vacation policy or expense reporting instead of searching a shared drive.

Result: New hires get accurate, sourced answers to routine policy questions in seconds, and the documents stay the single source of truth rather than being paraphrased incorrectly from memory.

 

⚖️ A Legal Team Searching Case Files

Hybrid search added specifically because exact terms matter

A legal team building a RAG tool over case files finds that pure vector search occasionally misses documents containing an exact statute number or case citation a lawyer searched for verbatim. Adding a keyword-based retriever alongside the vector retriever, combined through LangChain's ensemble retriever pattern, catches both semantic questions and exact-term lookups in the same system.

Result: A retrieval system that handles both 'find cases about breach of contract' and 'find the document citing Section 12.4' equally well.

 

🔧 Debugging Why Retrieval Quality Dropped

LangSmith tracing pinpointing a chunking problem after the fact

A support-documentation RAG app that worked well for months starts giving vague answers after a large batch of new, table-heavy documents was added. Tracing recent queries through LangSmith reveals the new documents' tables were being fragmented mid-row by the fixed-size text splitter, scattering related data across multiple disconnected chunks.

Result: A concrete diagnosis — switch to a format-aware splitter for tabular content — reached through tracing rather than guesswork.

9. Where to Go From Here

The pipeline built in this guide — load, split, embed, retrieve, generate, with conversational memory layered on top — covers the large majority of real RAG use cases without needing anything more exotic. A handful of natural next steps are worth knowing about once that foundation is solid.

•      LangGraph for more complex control flow: when a RAG app needs to conditionally decide whether to retrieve at all, retry a failed retrieval with a reformulated query, or route between multiple different retrievers, LangGraph — LangChain's graph-based orchestration layer — handles that branching logic more cleanly than a linear LCEL chain can.

•      Re-ranking retrieved chunks: adding a dedicated re-ranking model as a second pass after initial retrieval, before the top results go to the language model, is one of the most reliable ways to improve answer quality without changing your chunking or embedding strategy at all.

•      Structured evaluation datasets: building a set of representative questions with known-good answers, and running that set through LangSmith's evaluation tooling after every pipeline change, turns "does this feel better" into an actual measurable comparison.

The core lesson underlying every section of this guide is the same one experienced LangChain developers converge on repeatedly: the framework makes the easy 80% of a RAG app genuinely easy, and it's honest about the fact that the remaining 20% — chunking strategy tuned to your actual documents, retrieval quality tuned to your actual queries, and evaluation tuned to your actual users — is where the real engineering work lives, and where no framework can fully do the thinking for you.

ALSO READ THIS: Neural Networks Explained Visually

Conclusion — A Solid Foundation, Not a Finished Product

Building a working RAG app with LangChain takes a genuinely small amount of code — document loading, chunking, embedding, and an LCEL chain tying retrieval to generation, all of it covered in the sections above. That's real, and it's why LangChain has become the default starting point for so many RAG projects: the path from zero to a working prototype is short and well-documented.

What separates that prototype from something worth deploying is exactly what this guide spent its second half on: chunking that respects your documents' actual structure, retrieval that combines semantic and keyword search, conversational memory that handles real follow-up questions, and an evaluation loop that tells you when something's quietly gotten worse. None of that requires a different framework — it's the same LCEL foundation, extended in the specific directions your actual documents and actual users demand.

Have you built a RAG app with LangChain? What tripped you up first — chunking, retrieval, or something else? Share it in the comments below.

Comments

Popular posts from this blog

How Large Language Models (LLMs) Actually Work

What is Agentic AI? Complete Beginner Guide for 2026

Best Free AI Tools You Should Use Daily