RAG and agentic RAG explained: how AI agents find, evaluate, and use external knowledge

A language model can produce an answer that sounds informed, detailed, and confident. But where did the information come from? Was it learned during training, retrieved from a company document, taken from an outdated contract — or invented? Does it still apply today? Can anyone verify it?

These questions matter whenever AI is used for work that depends on facts. A commercial manager asking “should we run a 15% discount on family-size yogurt at Retailer A next quarter?” needs historical sales, comparable promotions, current margins, retailer fees, inventory, capacity, notice periods, and approval policies — information that changes over time, is private to the company, was never in the model’s training data, and is subject to access controls. The model needs external evidence. That is the purpose of retrieval-augmented generation: retrieve relevant information, place it in the model’s context, generate a grounded answer.

But complex business questions do not fit into one search. The system may need to clarify the question, identify entities, decompose it, choose among sources, inspect results, notice what is missing, resolve contradictions, search again, and decide when the evidence is enough — or abstain when it is not. That is agentic RAG: traditional RAG treats retrieval as a pipeline stage; agentic RAG treats retrieval as an investigation.

RAG gives a model access to external information. Agentic RAG gives an agent responsibility for deciding what information is needed, where to find it, whether it is trustworthy, and when the evidence is sufficient.

The original RAG research combined a generative model with a retriever over an external, non-parametric knowledge index, letting the model answer from both its learned parameters and retrieved passages. The motivation holds five ways: training knowledge has a boundary and facts like prices, contracts, and inventory change continuously; private enterprise knowledge was never in the training data; parametric knowledgeis difficult to update selectively, whereas an external store can add a document or replace a contract; model answers are difficult to trace, whereas RAG preserves sources, dates, and citations; and models produce unsupported information — which RAG reduces but does not eliminate. A RAG system can still retrieve the wrong source, misread it, ignore it, or attach the wrong citation. RAG improves the model’s information environment; it does not automatically guarantee truth.

A complete traditional system has two processes: an offline ingestion pipeline (sources → access validation → parsing → cleaning → chunking → metadata → embedding and indexing) and an online query pipeline (question → query understanding → retrieval → reranking → context assembly → generation → citation). Every stage influences the result: a strong model cannot compensate for broken ingestion, and a correct passage is useless if the generator ignores it. RAG quality is a systems problem.

Ingestion: from sources to index

It starts before any embedding. Source selectionasks not “what data can we index?” but “which information should this agent be allowed to use as evidence?” — and every source carries an authority rank (signed contract > approved policy > system of record > internal analysis > historical case > user statement > external webpage) and a lifecycle: owner, effective date, expiry, version, approval status. Access control belongs before ingestion: the dangerous architecture retrieves everything and asks the model not to reveal unauthorized content; the safe one applies identity and permissions first, so the model never receives what it may not disclose.

Parsing is not text extraction. A poorly parsed financial table keeps every value while destroying the relationships between scenarios, discounts, and contributions; a good parser preserves rows, columns, units, titles, and hierarchy — and for important structured data, querying the underlying database beats retrieving text scraped from a report. Cleaning removes repeated headers, navigation debris, stale drafts, and hidden instructions while preserving the original for audit.

Chunking— dividing content into retrievable units — is one of the most underestimated RAG decisions: it determines what the retriever can find, whether a fact stays connected to its conditions, and how citations display. The toolbox: fixed-size chunks (simple, splits meaning), overlapping chunks (protects boundaries, duplicates content), paragraph and section-aware chunking (carries the heading path — “Commercial Terms > Display Fees”), semantic chunking (meaning-based boundaries at compute cost), parent–child chunking (match on a small chunk, return the full clause with its exceptions and definitions), hierarchical summaries, entity- and event-centered organization, and table-, slide-, and multimodal-aware units. There is no universal chunk size: small improves precision and citation specificity, large preserves context and adds noise. Chunk empirically. Finally, metadata— entity IDs, dates, versions, status, sensitivity, access group, authority — is what lets retrieval filter to the right retailer, the signed contract, the current date, the permitted audience. Without it, semantic similarity happily returns the wrong retailer’s old draft.

Retrieval modes

• Dense vector search. Embeddings place similar meaning close together, matching paraphrases and concepts even when wording differs. But similarity does not prove correctness, authority, freshness, permission, or entity identity — “Retailer A display fee 2024” is highly similar to a query about 2026, and wrong.
• Lexical search. BM25-style keyword matching wins for exact identifiers: SKU codes, clause numbers, error messages, rare terms. Semantic search often fumbles “SKU-9821”; lexical search does not.
• Hybrid search. Combine both and rerank — the query “SKU-9821 cannibalization during yogurt promotions” needs keyword precision for the SKU and semantic breadth for the concept.
• Structured retrieval. “What is current usable inventory?” deserves a live ERP query, not a paragraph from last week’s report. Text-to-SQL is powerful and risky (wrong joins, missing filters, unauthorized tables) — prefer read-only access, approved semantic models, validation, and governed analytics tools for recurring metrics. API retrieval through business tools is often the most reliable route to current operational facts.
• Web retrieval. Essential for market developments and regulations; brings SEO spam, outdated pages, contradictions, and prompt injection. Source evaluation becomes mandatory.
• Graph retrieval and GraphRAG. Knowledge graphs answer relationship questions — ownership, dependencies, multi-hop connections. GraphRAG-style approaches extract entities, relationships, and community summaries to answer corpus-wide questions (“major risks across all supplier incident reports”) that passage lookup cannot. Not automatically better: graph construction adds extraction cost and error surface, and for “what does clause 4.2 say?” ordinary retrieval is simpler and better.

From query to evidence

The user’s wording is rarely the best retrieval query. Query understanding identifies intent, entities, dates, and required evidence — and entity resolutioncomes first, because retrieving information about the wrong entity produces a perfectly grounded, completely wrong answer. Query rewriting sharpens (“what happened with the yogurt promotion?” → the retailer, product, and quarter); expansion issues several related queries for recall; HyDE-style approaches retrieve with a hypothetical generated answer — useful as a retrieval aid, never as evidence. Initial retrieval optimizes recall over a wide candidate set; reranking then applies deeper judgment: a business-aware reranker puts the current signed contract above the semantically-more-similar archived draft, because relevance is multidimensional — right entity, right time, sufficient authority, permitted, specific, non-duplicative.

Context assembly is where RAG meets context engineering: how many passages, in what order, expanded to parents or summarized, contradictions visible, sources labeled. More retrieved context is not better — the goal is minimum sufficient evidence, packaged so the model can reason about authority as well as content:

Source 1
Document:  Retailer A Agreement 2026
Section:   4.2 Display Fees
Authority: Current signed contract
Effective: 1 January 2026

Source 2
System:    Promotion analytics
Records:   8 comparable promotions
Current to: 30 June 2026

Source 3
System:    ERP
Inventory: 3,400 usable units
Timestamp: 17 July 2026, 10:30

Grounded generation then needs explicit expectations: use only supplied evidence for factual claims, distinguish facts from assumptions, flag missing information, cite every material claim, never resolve contradictions silently, abstain when evidence is insufficient. Note the distinction: groundedness asks whether the answer follows from the supplied evidence; factual correctness asks whether it is true in the world. An answer can be faithfully grounded in an outdated document and still wrong today. And citations must be claim-level: a citation is correct only if it supports the specific claim, complete only if all material claims are covered — and provenance must survive transformation, because every hop from raw document to chunk to summary to worker output to final synthesis is a place where sources get lost.

A RAG system becomes agentic when the agent has meaningful control over retrieval decisions: whether retrieval is needed at all, how to decompose the question, which source and tool to use, how to rewrite the query, whether to retrieve again, how to handle weak results, when to stop, and whether to answer or abstain. Agentic RAG is not RAG plus a bigger model — it is RAG plus planning, tool selection, evaluation, and iterative control.

The agentic toolkit builds on research patterns worth knowing by name. Adaptive RAG selects no retrieval, single retrieval, or multi-step retrieval based on query complexity — because unnecessary retrieval adds cost, noise, and injection exposure. Self-RAG taught a model to decide when to retrieve and to critique retrieved passages and its own output; the durable idea is that retrieval should not happen indiscriminately and passages should not be accepted automatically. Corrective RAG grades retrieved evidence and triggers correction — filtering, rewriting, switching to web search — instead of generating confidently from weak results. Retrieval grading classifies passages (relevant, outdated, wrong entity, insufficient authority) to decide the next move. Beyond these: multi-query retrieval for varied terminology, iterative retrieval (search → learn the contract lives under Holding B → search again), recursive retrieval that follows references with depth limits and cycle detection, and multi-hop retrieval that connects facts no single passage contains.

Agentic web research is this loop over an open, adversarial evidence space, which makes source-quality evaluation central: primary versus secondary, official versus unofficial, author expertise, corroboration — and the crucial distinction between publication date and event date: a recent article may describe an old event, and an older official document may still be the governing standard.

The hardest agentic question is when to stop. Evidence may be sufficient when every required subquestion is covered by current authoritative sources, material contradictions are resolved, each claim can be cited, remaining uncertainty is disclosed, and further search has low expected value. Requirements are task-specific: a factual lookup needs one authoritative source; a strategic recommendation needs data, assumptions, scenarios, and risks; legal analysis needs primary sources and qualified human review. An explicit evidence coverage matrix prevents premature completion:

Value of information controls research cost: if the recommendation is robust across competitor scenarios, more competitor research is low-value; if profitability hinges on the retailer fee, retrieving the current fee is everything. When sources contradict— six weeks’ notice versus eight — the agent must not silently pick one: inspect effective dates, status, entity, and authority; the conflict may be temporal, contextual, an outdated draft, or entity confusion. If it cannot be resolved, state it, describe its impact, and escalate — unresolved contradiction is an output, not a reason to invent certainty. And negative evidenceneeds care: “no contract found” may mean wrong entity name, permission restriction, or indexing delay — “not found” is not “verified not to exist.”

Abstention is not failure. Confident unsupported generation is failure.

RAG versus its neighbours

A production agentic RAG stack layers a query planner, a retrieval router, a permission and policy layer, the retrieval engines, reranking and evidence grading, an evidence state, a retrieval controller (continue, redirect, clarify, stop), and a grounded generator. Evidence state is the key addition: the original question, identified entities, required evidence, queries attempted, accepted and rejected passages, contradictions, missing information, and remaining budget. Retrieval history records what happened; evidence state records what is currently established — without it, the agent repeats searches and forgets why evidence was rejected.

Freshness is its own discipline: index synchronization must handle updates, deletions, permission changes, and supersession (an updated document must not simply add a second competing version); effective dates matter more than publication dates; and caches need policies — inventory should not be cached like a historical research paper.

1. 01The answer is not in the corpus. Use another source, ask, or abstain — no retriever finds what is absent.
2. 02The source exists but was not indexed. Monitor ingestion completeness and failures.
3. 03Bad parsing. Indexed, but meaning destroyed. Evaluate parsing on real documents, tables, and scans.
4. 04Poor chunking. The fact is separated from its condition. Structure-aware and parent–child retrieval.
5. 05Missed exact identifiers. Semantic search fumbles the SKU. Lexical or hybrid retrieval.
6. 06Similar but wrong evidence. Wrong year, customer, or country. Entity resolution, metadata filters, business-aware reranking.
7. 07Stale evidence. The old version outranks the current one. Status, effective dates, supersession metadata.
8. 08Retrieval noise. Too many weak passages. Thresholds, deduplication, evidence budgets.
9. 09Retrieval duplication. Near-identical chunks crowd the context. Clustering and diversity.
10. 10Lost surrounding context. Relevant sentence, missing definitions. Expand to parent sections.
11. 11Hidden contradictions. Only the top-ranked source retrieved. Search for diverse sources; test contradiction retrieval.
12. 12The generator ignores the evidence. Grounded-generation instructions; evaluate claim support.
13. 13Overcopying. The answer repeats retrieved text without solving the problem. Separate extraction from synthesis.
14. 14Hallucinated citations. Generate citations from tracked source identifiers, not free-form memory.
15. 15Lost provenance. Summaries passed between agents without sources. Citations in every intermediate output contract.
16. 16Endless retrieval. More is always possible. Sufficiency definitions, budgets, stopping rules.
17. 17Premature stopping. One passage found, constraints unchecked. Evidence coverage matrix.
18. 18Source-selection error. Searching documents when a live system holds the answer. Better routing and source descriptions.
19. 19Unauthorized retrieval. Permissions before retrieval; test isolation.
20. 20Prompt injection through retrieved content. Untrusted-input treatment, least privilege, gated actions.
21. 21Knowledge-base poisoning. Source approval, integrity monitoring, ingestion controls.
22. 22Correct retrieval, wrong reasoning. Evaluate the reasoning and calculation layer separately.
23. 23Citation overconfidence. Users trust any answer with citations. Verify that each citation supports its claim.

RAG must be evaluated component by component: a poor answer may come from missing knowledge, bad parsing, bad retrieval, bad reranking, bad assembly, bad reasoning, or bad citation, and one overall score does not reveal which. Build a representative set including ambiguous, multi-hop, date-sensitive, contradictory, permission-restricted, table-based, unanswerable, and prompt-injection cases — not just happy paths. Retrieval evaluation uses recall and precision at K, mean reciprocal rank, and NDCG — plus the enterprise dimensions that matter more: entity accuracy, freshness accuracy, authority accuracy, and permission accuracy, where the acceptable unauthorized-retrieval rate is effectively zero. Generation evaluation covers faithfulness, answer relevance, completeness, claim-level citation correctness and completeness, and abstention accuracy — a system that always answers looks helpful while being unsafe.

Agentic evaluation adds query planning, routing accuracy, retrieval rounds, redundant searches, contradiction detection, sufficiency decisions, stopping behaviour, cost, and recovery — and trajectory evaluation: an agent that searched the web eighteen times, touched unauthorized data, and happened to produce the right answer did not succeed. Ultimately, measure the business outcome — resolution rate, research time, decision quality — and keep expert humans on source quality, legal interpretation, and high-stakes correctness.

1. 01Understand the goal. Objective (incremental contribution profit), entities (Retailer A, family-size yogurt, next quarter), and the ten required evidence categories from baselines to approval rules.
2. 02Route evidence needs. History → analytics database; margin → ERP; fees and notice → contract repository; capacity → planning system; policy → knowledge base; competitors → market intelligence. Not one broad query to one vector index.
3. 03Resolve entities. The contract search returns nothing for Retailer A; memory suggests Holding B; the master-data system confirms; the search is reformulated.
4. 04Retrieve comparables. 42 promotions filtered to 8 strong comparables: median uplift 22%, cannibalization 14%, three of eight negative. Raw records preserved for audit.
5. 05Retrieve retailer terms. Current signed agreement: €68,000 display fee, eight weeks’ notice. The old €55,000 agreement is graded historical and superseded.
6. 06Run the financial analysis. The 15% scenario: −€30,000 to +€3,000 expected contribution. The proposal is not attractive.
7. 07Reformulate. New question: which lower discount produces positive contribution? 8%, 10%, and 12% evaluated; 10% wins on risk-adjusted contribution.
8. 08Check feasibility. Inventory is short at the upper demand scenario; capacity is available only if reserved before 20 August. The recommendation becomes conditional.
9. 09Check sufficiency. The coverage matrix shows everything complete except cannibalization (moderate uncertainty, disclosed) and competitor activity (missing, unlikely to change the conclusion).
10. 10Produce a cited recommendation. Do not run 15%; run 10% conditional on capacity reservation — every claim linked to the analytics result, current contract, ERP record, capacity plan, and commercial policy.

When is each mode right? Traditional RAGsuffices for policy Q&A, manual lookups, and FAQ assistants — known source, one retrieval round, low risk. Agentic RAG earns its cost for market research, due diligence, promotion planning, supplier risk, and incident investigation — multiple sources, likely contradictions, and the need to know when not to answer. And sometimes neither: when direct generation works, when one deterministic query answers the question, or when latency rules out iteration, a fixed tool call beats an autonomous research loop.

Most organizations should stabilize Levels 2 and 3 before adding broad agentic autonomy. TiMiNa’s RETRIEVE framework then structures the design:

1. 01Resolve the question and entities. Intent, entities, period, geography. Wrong entity resolution invalidates everything downstream.
2. 02Establish the required evidence. Define what must be known before answering — an evidence contract.
3. 03Target the correct sources. Route each need to its source of truth; one vector index is not the answer to every information problem.
4. 04Retrieve with the appropriate method. Lexical, semantic, hybrid, SQL, API, graph, or web — fit the method to the information.
5. 05Inspect relevance, authority, and freshness. Right entity, current version, sufficient authority, permitted access.
6. 06Expand or correct the search. Rewrite, search aliases, expand to parents, change source, decompose, ask the user.
7. 07Verify claims and contradictions. Material claims supported, conflicts resolved or disclosed, citations matching claims.
8. 08End when evidence is sufficient. Then answer, answer conditionally, abstain, or escalate.

Ten principles for RAG and agentic RAG

1. 01Start with the source of truth. Do not begin with embeddings; begin with authoritative information.
2. 02Retrieval quality begins before retrieval. Sources, parsing, chunking, metadata, and permissions shape what can be found.
3. 03Semantic similarity is not truth. A similar passage may be outdated, unauthorized, or about the wrong entity.
4. 04Use the right retrieval mode for the information. Documents, SQL, graphs, APIs, and the web solve different problems.
5. 05Preserve provenance throughout the system. Every material claim stays traceable to evidence.
6. 06Treat retrieved content as untrusted input. External text may contain errors, manipulation, or hostile instructions.
7. 07Search iteratively only when the task requires it. Agentic retrieval adds flexibility — and cost, latency, and risk.
8. 08Make evidence sufficiency explicit. Do not stop merely because something relevant was found.
9. 09Evaluate retrieval and generation separately. A correct final answer does not prove the pipeline is reliable.
10. 10Design for abstention. A trustworthy system knows when it lacks enough evidence.

What is agentic RAG?

A retrieval system in which an agent dynamically decides whether retrieval is needed, what evidence is required, where to search, how to reformulate queries, whether to search again, and when evidence is sufficient.

Is RAG just vector search?

No. Vector search is one retrieval method. RAG can also use keyword search, SQL, APIs, graphs, and web search — and a vector database by itself is not a RAG architecture.

Why is chunking important?

Chunking determines the units that can be retrieved. Poor chunks separate facts from their conditions, damage tables, and lose document structure — making a good model look incapable.

What is reranking?

A second-stage process that evaluates retrieved candidates more carefully — relevance, entity, authority, freshness, permissions — and reorders them before they enter the model’s context.

Does RAG eliminate hallucinations?

No. It reduces unsupported generation, but errors still occur through bad sources, retrieval mistakes, incorrect reasoning, and citation failures.

Is RAG better than fine-tuning?

They solve different problems. RAG suits current, private, changing, source-sensitive knowledge; fine-tuning suits stable behaviour, style, and specialized task performance. Many systems use both.

Is RAG better than a long context window?

Not always. Long context works for small corpora and global document analysis. RAG wins for large, dynamic, permissioned corpora where only a subset is relevant and citations are required.

Should every AI agent use RAG?

No. Retrieve when external evidence is needed. Unnecessary retrieval adds cost, latency, noise, and security exposure.

What is the biggest RAG mistake?

Treating RAG as “put documents in a vector database and connect an LLM” while ignoring source quality, chunking, metadata, authority, permissions, evaluation, and evidence sufficiency.

RAG solves one of the central limitations of language models: they are powerful generators and reasoners that do not automatically possess current knowledge, private company information, live operational data, or reliable citations. A traditional system retrieves, supplies context, and generates. An agentic system understands the goal, plans the evidence, selects sources, retrieves, evaluates, identifies gaps, searches again, resolves contradictions, verifies sufficiency, and answers or abstains. For complex agentic work, retrieval is not a database lookup — it is a disciplined process of evidence acquisition.

The pieces of this series each play their part: the model provides language and reasoning, the retrieval system provides evidence, the agent plans the investigation, the harness enforces access and control, context engineering determines what the model sees, memory preserves useful prior experience, and evaluation determines whether the whole system works. The defining question is not “did the system retrieve some related text?” but: did it obtain the right evidence, from the right source, for the right entity and time period, under the right permissions — and use it faithfully to support the decision?

The most advanced retrieval system is not the one that searches the most sources. It is the one that knows where to look, what to trust, what to reject, when to search again, when it has enough — and when it should not answer.