Fine-Tuning vs. Prompting

Fine-tuning adapts a pre-trained model to your specific domain or task. Understanding when it is worth the investment, and what alternatives exist, is essential for production AI decisions.

What fine-tuning is

Fine-tuning continues training a pre-trained model on a smaller, domain-specific dataset. The model updates its weights to better reflect the patterns in your data, improving performance on your specific task distribution. Unlike prompting, fine-tuned behaviour is baked into the model weights rather than provided at inference time.

When fine-tuning makes sense

• You have a narrow, well-defined task where format and style consistency are critical
• You have enough high-quality training examples (typically hundreds to thousands)
• You need the model to reliably follow a specific output schema that prompting alone does not achieve
• Latency or cost constraints prevent including extensive examples in every prompt
• You need the model to internalise proprietary terminology or domain-specific knowledge

When prompting (or RAG) is better

Fine-tuning is not a solution for hallucinations. The model cannot reliably internalise factual knowledge through fine-tuning and retrieve it accurately. For tasks requiring accurate recall of specific facts or documents, retrieval-augmented generation (RAG) is almost always superior.

Practical costs

Fine-tuning on proprietary APIs (OpenAI, Anthropic) costs money and requires careful data preparation. The operational overhead of maintaining training datasets, tracking model versions and managing deployment should be weighed against the performance gains versus a well-prompted base model.

Fine-tuning should come after prompting has been optimised, not instead of it. Many problems attributed to needing fine-tuning can be solved with better prompts and few-shot examples.