What RAG does
Retrieval-augmented generation searches your own content first — policies, manuals, tickets, product data — and gives the most relevant passages to the model along with the question. The model writes its answer from that material and can cite where each point came from.
Because the knowledge lives in your documents, updating the answer is as simple as updating the document. No retraining is needed.
What fine-tuning does
Fine-tuning further trains a model on examples of the inputs and outputs you want. It changes how the model behaves: its tone, the structure of its replies, or how well it handles a narrow, repetitive task.
Fine-tuning is not a good way to teach facts that change. Facts learned in training go stale, and the model cannot show where they came from.
A quick way to decide
- Answers must reflect your latest documents → RAG.
- Users need to see sources → RAG.
- Different users may see different documents → RAG with permission filters.
- Output must follow a strict format or house style → fine-tuning, or strong prompting first.
- High volume of one narrow task where cost matters → fine-tuning a smaller model can help.
At a glance
| RAG | Fine-tuning | |
|---|---|---|
| Best for | Answers from your documents | Style, format, narrow skills |
| Keeps facts current | Yes — update the document | No — needs retraining |
| Shows sources | Yes | No |
| Respects user permissions | Yes, with filters | No |
| Time to first version | Days to weeks | Weeks |
Tip: try strong prompting and retrieval first. Fine-tune only when measurements show a gap that prompting cannot close.
Why most teams start with RAG
RAG is quicker to build, easier to update and easier to audit. Good prompts plus good retrieval solve the majority of business question-answering needs. Fine-tuning is a later optimisation, added when measurements show a specific gap.
What makes RAG work well
- Clean source documents with clear structure
- Sensible chunking so passages keep their meaning
- Hybrid search that combines meaning and keywords
- A test set of real questions to measure retrieval and answer quality