RAG vs fine-tuning for a support bot: which one do you actually need?

Every team building a support bot hits the same fork in the road. Should the model be trained on your content, or should it look your content up while it answers? The words sound interchangeable in a pitch deck. They are not, and picking the wrong one costs you months.

Here is the short version: if your goal is for a bot to answer questions about your product, policies, and prices, you want retrieval. Fine-tuning solves a different problem than the one you have.

What each one actually does

Fine-tuning continues training a base model on examples you supply. The result is a new set of weights. Knowledge you feed it becomes diffuse — spread across billions of parameters with no address, no timestamp, and no way to point at where an answer came from.

Retrieval (often called RAG) leaves the model alone. Your content sits in a search index. Each question triggers a search; the passages that come back are handed to the model with an instruction to answer from them and nothing else.

The distinction that matters is not accuracy in the abstract. It is where the knowledge lives — and therefore what happens when that knowledge changes.

The test that settles it: what happens on Tuesday?

On Tuesday you change your refund window from 30 days to 14.

With retrieval, you edit the page. The next crawl picks it up. The next visitor gets 14 days, with a link to the page that says so. Elapsed effort: editing one page.

With fine-tuning, the model still believes 30 days. It will say so confidently, because from the inside there is no difference between a fact learned last month and a fact that is still true. To fix it you assemble a new training set, run a new fine-tune, evaluate it, and deploy it — and you repeat that every time any content changes. Meanwhile the wrong answer keeps shipping.

For a support bot, content changing is not an edge case. It is the entire job.

Three more things retrieval gives you that weights cannot

Citations. Because the answer was built from specific passages, you can show which ones. A visitor clicks and verifies; you get an audit trail when something looks wrong. A fine-tuned model cannot tell you why it said something, because it does not know.

An honest failure mode. Retrieval can come back empty, and empty is a checkable state. That is what allows a bot to say "I couldn't find that" instead of composing something plausible. A fine-tuned model has nothing to notice and no way to abstain.

Per-tenant isolation. If you serve many customers, retrieval keeps each one's content in its own index — a hard boundary you can point at in a security review. Fine-tuning would need one model per customer, which is neither affordable nor separable.

So when is fine-tuning right?

It is a real technique with real uses. It is just not a knowledge store:

• Format and tone, when you need consistent output shape and prompting is not getting you there.
• A narrow classification task you run millions of times, where a small tuned model is cheaper than a large prompted one.
• A domain vocabulary the base model handles badly — unusual jargon, a low-resource language.

Notice that none of these are "the model should know our prices." Style is a property of the weights. Facts are not.

The hybrid people actually ship

In practice the sensible setup is: retrieval for everything factual, prompting for voice, and fine-tuning only if you have measured a specific gap that neither closed. Most teams never reach the third step, and that is a good outcome rather than a compromise.

What you should be measuring is not "did we fine-tune." It is:

• What share of answers were grounded in real retrieved content?
• How many questions did the bot decline, and did those become new pages?
• When an answer was wrong, could you trace it to the passage that misled the model?

Retrieval makes all three of those questions answerable. That is the whole argument.

The practical takeaway

If someone is proposing to fine-tune a model on your help centre, ask them what happens when a page changes. The answer will tell you whether they are solving your problem or an adjacent, more interesting-sounding one.

Embent takes the retrieval route end to end: our crawler reads your pages, answers are built from what it finds, every claim carries a citation, and anything the search cannot support becomes an honest "I don't know" plus a logged question for you to write about.