If you need to move a pallet across a warehouse, you can technically use a helicopter. It will be memorable. People will take photos. Finance will eventually find you.

Model selection has a helicopter problem. The largest model wins the public benchmark, so it becomes the default for invoice classification, autocomplete, routing, extraction, and every other task that could fit on a mildly ambitious spreadsheet.

Do not rent a helicopter

Large models are valuable because they handle ambiguity and unfamiliar problems. But production traffic is often repetitive. The same document types arrive every day. The same fields need extraction. The same twenty intents cover nearly every support message.

That repetition is an opportunity. Microsoft's Phi-3 technical report showed how much capability can fit into a 3.8-billion-parameter model small enough for constrained devices. The interesting lesson is not that one small model replaces every large one. It is that size stopped being a useful synonym for deployable quality.

Boring is a workload

“Boring” has technical properties:

  • the input distribution is narrow and changes slowly,
  • the desired output has a strict schema,
  • failures can be detected cheaply,
  • examples are plentiful, and
  • latency or privacy matters on every request.

This is ideal territory for a small model, sometimes with task-specific fine-tuning. Running locally may remove a network round trip. A smaller footprint permits more replicas, simpler autoscaling, and better isolation. At high volume, “slightly cheaper” stops being an adjective and becomes a headcount.

Fit beats rank

Do not compare models on an average benchmark and declare a winner. Build a test set from the actual boring job. Include ugly scans, empty fields, unusual languages, and the customer who enters a paragraph where the form asked for a ZIP code.

Then measure the whole system: exactness, tail latency, cost, memory, privacy boundary, and how easily failure can be recognized. A small model that is 98% correct and knows when to escalate can beat a larger model that is 99% correct but costs ten times more and sometimes fails poetically.

Design a promotion path

The best architecture is rarely “small model only.” Let the specialist handle familiar traffic and promote the hard cases:

request → small model → validate
                         ├─ pass → return
                         └─ fail → larger model or human

Now the expensive model spends its budget on ambiguity instead of reformatting dates. The small model gets the boring job. That is not a consolation prize. In production, boring is where most of the work lives.

Further reading: Abdin et al., Phi-3 Technical Report.