Few-shot prompting is magic—until it blows up in your face. Imagine you build a customer sentiment classifier and give it 3 examples. In all 3 of your examples, the customer's name happens to be 'John', and all 3 reviews are about 'laptops'.
Now you send in real customer data about an angry customer named 'Sarah' complaining about 'shipping delays'. To your horror, the AI outputs: 'John had an issue with laptop shipping.'
The Accidental Pattern Trap
AI models are hyper-sensitive pattern detectors. If all your examples share an accidental similarity, the model doesn't know it was an accident—it assumes it is a hard mathematical law.
- Date Bias: All examples use
MM/DD/YYYY, so the AI breaks when input hasYYYY-MM-DD. - Length Bias: All examples are exactly 12 words, so the AI truncates complex answers to 12 words.
- Order Bias: The first example is always positive and the second is always negative, so the AI alternates outputs regardless of truth.
The Orthogonal Variation Rule
When writing 3 examples, deliberately vary every single non-essential detail: vary sentence lengths, vary entity genders, vary industries, and vary date formats. Teach the underlying logic, not surface mimicry.