Temperature

Temperature is a numerical parameter (typically ranging from 0 to 2) that controls how random or deterministic a language model's output will be. At low temperatures (near 0), the model consistently chooses the most probable next token, producing focused, predictable outputs. At higher temperatures, the model is more willing to select less probable tokens, leading to more creative, varied, and sometimes surprising responses. The name comes from thermodynamics, where higher temperatures correspond to more energetic, random molecular behaviour. In AI, the analogy is apt: higher temperature means more "energetic" exploration of different word choices and phrasings.

When a language model generates text, it calculates a probability distribution over its vocabulary for each token. Temperature modifies this distribution before a token is selected. At temperature 0, the model always picks the highest-probability token (greedy decoding). As temperature increases, the probability distribution becomes flatter, giving lower-probability tokens a better chance of being selected. For example, if a model is completing "The weather today is..." with probabilities: sunny (40%), warm (30%), cloudy (20%), unpredictable (10%), at temperature 0 it always outputs "sunny." At temperature 0.7, it might occasionally choose "warm" or "cloudy." At temperature 1.5, even "unpredictable" becomes a realistic choice. Temperature is often used alongside other sampling parameters like top-p (nucleus sampling), which limits selection to the smallest set of tokens whose cumulative probability exceeds a threshold. Together, these parameters give fine-grained control over output diversity.

Choosing the right temperature is essential for matching AI outputs to business requirements. Factual tasks — data extraction, classification, analysis, code generation — benefit from low temperatures (0-0.3) that prioritise accuracy and consistency. Creative tasks — marketing copy, brainstorming, content ideation — benefit from higher temperatures (0.7-1.0) that encourage variety and originality. Temperature also affects reproducibility. At temperature 0, the same input produces the same output every time, which is important for testing, auditing, and compliance. At higher temperatures, outputs vary between runs, which is desirable for creative applications but problematic for deterministic workflows. For production applications, temperature is a tunable parameter that should be tested and optimised for each specific use case rather than set to a single default value across all applications.

A practical starting framework: use 0-0.2 for factual question answering, data extraction, classification, and structured output generation. Use 0.3-0.6 for general-purpose tasks like summarisation, explanation, and customer support. Use 0.7-1.0 for creative writing, brainstorming, and content generation. Values above 1.0 are rarely useful in production as they tend to produce incoherent or nonsensical outputs. The optimal temperature also depends on the model being used — some models are calibrated differently, and the same temperature value may produce different levels of randomness across models. Testing with representative inputs and evaluating output quality at different settings is the most reliable way to find the right value for your application.