Few-Shot Learning

Few-shot learning is the practice of providing a language model with a small number of example input-output pairs in the prompt before asking it to perform a task. These examples demonstrate the desired behaviour — format, style, reasoning approach, or classification criteria — allowing the model to understand the pattern and replicate it for new inputs. For instance, rather than describing in abstract terms how to categorise support tickets, you provide three examples of tickets with their correct categories. The model learns the categorisation logic from these examples and applies it to new tickets. This approach consistently outperforms zero-shot prompting (no examples) for most tasks.

Few-shot learning leverages a capability called in-context learning — the model's ability to recognise and extend patterns presented within its input context. When the model sees examples of a task, it effectively "programs" itself for that task within the current conversation, without any permanent changes to its parameters. The quality of few-shot performance depends on several factors. Example selection matters — representative, diverse examples that cover edge cases produce better results than similar examples. Example ordering can also affect results, with some research suggesting that placing more complex examples later improves performance. The number of examples needed varies by task complexity. Simple classification tasks may need only 2-3 examples, while complex extraction or reasoning tasks may benefit from 5-10. Beyond about 10 examples, returns diminish and fine-tuning may be more efficient.

Few-shot learning occupies a sweet spot between zero-shot simplicity and fine-tuning complexity. It requires minimal setup — just a handful of examples — yet delivers substantial accuracy improvements for most tasks. This makes it the most practical approach for many business applications. For organisations, few-shot learning enables rapid deployment of customised AI capabilities. A new document classification system can be operational within hours, requiring only a few representative examples rather than thousands of labelled training samples. Changes are equally quick — updating the examples immediately changes the model's behaviour. Few-shot learning also provides transparency. Stakeholders can review the examples to understand exactly what the model has been shown and what behaviour it is expected to replicate. This makes it easier to audit, adjust, and explain AI behaviour compared to fine-tuned models where the training data influence is less visible.

Few-shot learning is used across a wide range of business applications. In customer support, examples demonstrate how to classify, prioritise, and respond to different ticket types. In content creation, examples establish the desired tone, format, and style. In data extraction, examples show what information to pull from documents and how to structure it. Few-shot is particularly effective for tasks where the desired output format is specific or unusual. Rather than describing a complex output format in words, a few examples make the expected format immediately clear to the model. This is commonly used for structured data extraction, report generation, and data transformation tasks.