RAG, fine-tuning or an agent? What your AI feature actually needs

Ask three vendors how to add AI to your product and you will hear three answers: RAG, fine-tuning and agents. They are not rivals. They solve different problems, and picking the wrong one is how AI features end up expensive and still wrong.

RAG: give the model your facts

Retrieval-augmented generation finds the relevant parts of your documents, records or catalogue and hands them to the model with the question. The model answers from your facts instead of its memory. Use it when answers depend on information that is yours, changes often, or must be quoted correctly: policies, product details, past tickets, contracts.

Fine-tuning: change how the model behaves

Fine-tuning trains a model further on examples, so it learns a format, a tone or a narrow task. It does not reliably teach the model new facts, and every time your facts change you would have to train again. Use it when you have many good examples of a very specific output and prompting alone cannot get there.

Agents: let the model take steps

An agent uses a model to decide what to do next: search, call a tool, check the result, try again. Use it when the job is a sequence of steps that depends on what each step finds, not a single answer.

What most business features need

• Questions over your own documents or data: RAG.
• The same task done in a fixed way, such as extracting fields from invoices: a good prompt and validation, sometimes fine-tuning later.
• A job with several steps and tools, such as qualifying a lead and booking a call: an agent, using RAG for its facts.

Start with RAG and a clear prompt, measured against real examples. Most teams never need fine-tuning, and the ones who do know it because the tests told them.

Where RAG goes wrong

Most RAG failures are data failures, not model failures: stale documents, duplicates that disagree, scanned PDFs nobody can search, or permissions ignored so one customer can retrieve another's records. Gartner warned in 2025 that organisations will abandon AI projects not supported by AI-ready data. Cleaning and structuring the sources is usually the biggest part of the job, and the part that decides whether answers are right.