Pattern recognition
Chapter 2 of 5

Pattern recognition

Expert intuition is mostly fast recognition: a large library of patterns, each linked to what it means and what to do. This chapter shows how that library is built, when it deserves your trust, how it fools you, and how to train it on purpose.

About 30 minutes38 sourcesInteractive labs and a quiz
80%+
of fire commanders’ decisions came from recognising a familiar situation, not comparing options
Source 8: Klein et al., 2010
94%
of 136 studies found a simple formula predicted as well as or better than experts
Source 20: Grove et al., 2000
61% vs 35%
accuracy on new paintings after mixed versus blocked study of painters
Source 35: Kornell & Bjork, 2008
250 ms
enough for radiologists to sense an abnormal scan above chance, without knowing where
Source 14: Evans et al., 2013

The big idea

When you read the word "cat", you do not sound it out letter by letter. You see it in one glance, because your brain stores that combination of letters as a single unit. You built that unit years ago, through thousands of encounters with the word.

Pattern recognition is this ability applied to everything: noticing that what you see, hear or handle right now matches a structure you have met before, and using that match to decide what it is, what will happen next and what to do. Infants use it to find words in a stream of speech. A chess master uses it to see "a weak king position" where a beginner sees 25 separate pieces. A radiologist uses it to sense that a mammogram is wrong before finding anything specific.

A large body of research suggests that much of what we call expert intuition is fast pattern recognition. Experience builds a large internal library of patterns, and each pattern is linked to what it means and what to do about it [2Primary studyPerception in chessChase & Simon, 1973 · Cognitive Psychology 4:55-81, 7Primary studyRapid decision making on the fire groundKlein et al., 1986 · Proceedings of the Human Factors Society Annual Meeting 30:576-580, 18ReviewConditions for intuitive expertise: a failure to disagreeKahneman & Klein, 2009 · American Psychologist 64:515-526].

This chapter makes two points that matter for any learner:

  1. Intuition can be trained. The right kind of practice, with clear feedback, builds your pattern library faster.
  2. Intuition is only as good as the world that trained it. In some fields, gut feelings become reliable. In others, the same mental machinery produces confident mistakes.

What experts actually see

The chess studies

In the 1940s, Dutch psychologist and chess master Adriaan de Groot studied how grandmasters choose moves. He expected them to calculate more moves ahead than weaker players. They did not. The difference was in what they saw: after a glance of a few seconds, grandmasters could rebuild a game position almost perfectly [1BookThought and Choice in Chess (2nd ed.; Dutch original 1946)de Groot, 1978 · De Gruyter Mouton].

In 1973, William Chase and Herbert Simon pushed this further [2Primary studyPerception in chessChase & Simon, 1973 · Cognitive Psychology 4:55-81]. A master, a strong club player and a beginner each looked at a chessboard for 5 seconds and then rebuilt it from memory. Sometimes the position came from a real game. Sometimes the same pieces were scattered at random.

How many pieces could each player rebuild after a 5-second look?
Pieces placed correctly on the first try, middle-game positions with 24 to 26 pieces. One player per skill level.
about 16
3 to 4
Master
about 8
3 to 4
Club player
about 4
3 to 4
Beginner
Real game positionRandom position
Sources: Chase & Simon, 1973 [2Primary studyPerception in chessChase & Simon, 1973 · Cognitive Psychology 4:55-81]; random-board summary from Simon & Chase, 1973 [3ReviewSkill in chessSimon & Chase, 1973 · American Scientist 61:394-403]. A small study: treat the exact numbers as illustrative.

On real positions, recall rose steeply with skill. On random positions, the advantage almost disappeared: everyone recalled about 3 to 4 pieces [3ReviewSkill in chessSimon & Chase, 1973 · American Scientist 61:394-403]. So the master did not have a better memory in general. The master had a better memory for chess as it is actually played.

Later work refined this. Across many studies, strong players do keep a small advantage even on random boards, about one extra piece per 400 rating points, compared with about five extra pieces per 400 points on real positions [5ReviewRecall of rapidly presented random chess positions is a function of skillGobet & Simon, 1996c · Psychonomic Bulletin & Review 3:159-163]. And when positions were mirror-flipped, recall dropped, which shows that experts' stored patterns are tied to specific squares, not just abstract ideas [4Primary studyRecall of random and distorted chess positions: implications for the theory of expertiseGobet & Simon, 1996b · Memory & Cognition 24:493-503].

Chunks: the unit of expertise

Short-term memory is small. It holds only a handful of items at once. Chase and Simon proposed that experts get around this limit by changing what counts as one item. A beginner must store six separate pieces. A master stores "castled king with the usual pawn shield" as one chunk [2Primary studyPerception in chessChase & Simon, 1973 · Cognitive Psychology 4:55-81]. Computer simulations support an estimate that a chess master holds roughly 50,000 such chunks [4Primary studyRecall of random and distorted chess positions: implications for the theory of expertiseGobet & Simon, 1996b · Memory & Cognition 24:493-503].

Try it yourself. The letters are the same in both rounds. Only the arrangement changes.

Interactive lab

Feel what a chunk is

You will see 18 letters for five seconds, then type as many as you remember, in order. Try the random set first, then the structured set. Both use exactly the same letters.

Random: not triedStructured: not tried

Ready? The letters disappear after five seconds.

Templates: bigger, flexible chunks

Chunking theory had a problem: experts can remember far more than a handful of chunks would allow. In one experiment, a chess master trained to recall several boards at once and reached nine boards with over 70% accuracy, up to 160 pieces [6Primary studyTemplates in chess memory: a mechanism for recalling several boardsGobet & Simon, 1996a · Cognitive Psychology 31:1-40]. The explanation is templates: frequently met patterns grow into larger structures in long-term memory, with a fixed core and open "slots" that can be filled quickly with the details of the current situation. Think of a printed form where the expert only writes in what is different this time.

Recognition drives action: the firefighter studies

Patterns matter because each one is linked to an action. Psychologist Gary Klein interviewed 26 experienced fire commanders, with an average of 23 years on the job, about 156 decisions they made in difficult incidents [8Primary studyRapid decision making on the fire ground: the original study plus a postscriptKlein et al., 2010 · Journal of Cognitive Engineering and Decision Making 4:186-209]. He expected to find them comparing options. They rarely did. In fewer than 12% of decisions was there evidence of comparing two or more options. In over 80%, the commander recognised the situation as a familiar type and acted on the option that came with that recognition.

Klein called this the recognition-primed decision model [7Primary studyRapid decision making on the fire groundKlein et al., 1986 · Proceedings of the Human Factors Society Annual Meeting 30:576-580]. Experienced people:

  1. recognise the situation as a type they know,
  2. get a workable action along with the recognition, so the first option is usually good enough,
  3. check it by imagining how it would play out, and change it only if the imagined version fails.

Herbert Simon summed up the idea by arguing that intuition is essentially recognition [9ReviewWhat is an "explanation" of behavior?Simon, 1992 · Psychological Science 3:150-161]. It is fast because the hard work was done earlier, over years of exposure.

How the brain builds patterns

Statistics without instruction

Pattern learning starts before we can talk. In a famous experiment, 8-month-old infants listened to a two-minute stream of made-up syllables with no pauses, where some syllables always followed each other (like the sounds inside a word) and others did not (like sounds across a word boundary). Afterwards, the infants treated the "words" differently from sequences that crossed boundaries [10Primary studyStatistical learning by 8-month-old infantsSaffran et al., 1996 · Science 274:1926-1928]. They had picked up the statistics of the stream without any teaching.

A later meta-analysis found this effect is real but small and sensitive to how natural the speech sounds [11Meta-analysisQuantifying infants' statistical word segmentation: a meta-analysisBlack & Bergmann, 2017 · Proceedings of the 39th Annual Conference of the Cognitive Science Society, 124-129]. The ability exists from infancy, but it is not as dramatic as the original paper suggested.

The brain as a prediction machine

A leading theory, predictive processing, holds that the brain constantly predicts what it is about to see, hear and feel, and only passes the surprises (prediction errors) up the chain to update its model [12ReviewWhatever next? Predictive brains, situated agents, and the future of cognitive scienceClark, 2013 · Behavioral and Brain Sciences 36:181-204]. On this view, recognising a pattern means having a model that predicts the input well. A coastal weather forecaster with 20 years of experience is not surprised by the afternoon sea breeze, but notices immediately when it does not arrive. Expertise is a sharper model. The theory is influential but not settled; even its leading advocate lists open challenges.

Experts literally see differently

Perceptual learning means experience changes how you extract information, not just what you remember [13ReviewPerceptual learning and human expertiseKellman & Garrigan, 2009 · Physics of Life Reviews 6:53-84]. Two things improve: you discover which features actually matter, and you pick them up faster and with less effort. Most teaching focuses on facts and procedures you can put into words, while much of expertise depends on this kind of seeing, which ordinary lessons leave untrained.

Two demonstrations stand out:

  • The gist of the abnormal. Radiologists and cytologists saw medical images for as little as a quarter of a second. They could tell normal from abnormal above chance, yet could not say where the problem was [14Primary studyThe gist of the abnormal: above-chance medical decision making in the blink of an eyeEvans et al., 2013 · Psychonomic Bulletin & Review 20:1170-1175]. They were reading an overall texture of "something is off".
  • Expert brains specialise. People trained to become experts at telling apart novel creature-like objects called "Greebles" showed increased activity in a brain area usually linked to faces [16Primary studyActivation of the middle fusiform "face area" increases with expertise in recognizing novel objectsGauthier et al., 1999 · Nature Neuroscience 2:568-573], and real car and bird experts showed the same for their specialty [17Primary studyExpertise for cars and birds recruits brain areas involved in face recognitionGauthier et al., 2000 · Nature Neuroscience 3:191-197]. Whether this area is special to faces or to expert recognition in general is still debated.
How well can radiologists spot an abnormal mammogram from a glimpse?
Area under the curve (AUC) by viewing time. 0.50 is pure chance, 1.00 is perfect.
250 ms0.64
500 ms0.65
1,000 ms0.66
2,000 ms0.72
Source: Evans et al., 2013 [14Primary studyThe gist of the abnormal: above-chance medical decision making in the blink of an eyeEvans et al., 2013 · Psychonomic Bulletin & Review 20:1170-1175]. A real signal, but far weaker than a full expert read.

The gist is real but weak. The researchers stress that it falls far short of what the same experts achieve with a full examination, and a follow-up found that looking longer did not make the first impression stronger [15Primary studyComparable prediction of breast cancer risk from a glimpse or a first impression of a mammogramRaat et al., 2021 · Cognitive Research: Principles and Implications 6:72]. A first impression is a hypothesis to check, not a diagnosis.

When can you trust your gut?

In 2009, psychologist Daniel Kahneman, famous for documenting the flaws of intuition, and Gary Klein, famous for documenting its strengths, wrote a joint paper to settle their disagreement [18ReviewConditions for intuitive expertise: a failure to disagreeKahneman & Klein, 2009 · American Psychologist 64:515-526]. They found they mostly agreed. Intuition deserves trust only when two conditions hold:

  1. The environment is regular enough to be predictable. There are real patterns linking cues to outcomes.
  2. The person has had a real chance to learn those patterns, through long practice with fast, accurate feedback.

They added a warning that applies to everyone: feeling confident is not evidence of being right. Confidence comes from how easily an answer arrives, not from whether it is correct.

Kind and wicked environments

Psychologist Robin Hogarth gave this idea a memorable name [19ReviewThe two settings of kind and wicked learning environmentsHogarth et al., 2015 · Current Directions in Psychological Science 24:379-385]. In a kind learning environment, feedback is quick, accurate and complete, and the situations you learn in match the ones you face later. In a wicked environment, feedback is delayed, missing or misleading, so experience can teach the wrong lessons. Kahneman and Klein point to chess players, firefighters and weather forecasters as people whose fields are regular enough to build real intuition, and to stock pickers and long-term political forecasters as people whose fields are not [18ReviewConditions for intuitive expertise: a failure to disagreeKahneman & Klein, 2009 · American Psychologist 64:515-526].

Kind environmentCues you noticeYour judgmentWhat really happensFast, accurate feedbackWicked environmentCues you noticeYour judgmentWhat really happensFeedback late, partial or misleading
Kind versus wicked. Concept diagram: what the feedback loop looks like in each kind of environment.

Formulas often beat experts in wicked fields

In a meta-analysis of 136 studies in psychology and medicine, researchers compared human experts with simple formulas given the same information [20Meta-analysisClinical versus mechanical prediction: a meta-analysisGrove et al., 2000 · Psychological Assessment 12:19-30].

Who predicts better: expert judgment or a simple formula?
Share of 136 studies in psychology and medicine
Formula clearly better47%
About equal48%
Expert clearly better6%
Source: Grove et al., 2000 [20Meta-analysisClinical versus mechanical prediction: a meta-analysisGrove et al., 2000 · Psychological Assessment 12:19-30].

The formula's advantage held regardless of how experienced the judges were. In 7 of the 8 studies where experts won, they had access to more information than the formula. This does not mean expertise is worthless. It means that in noisy fields, a checklist or formula is a strong baseline, and human judgment adds most when it brings information the formula does not have.

Forecasting: foxes, hedgehogs and superforecasters

Political scientist Philip Tetlock tracked experts' predictions about world events for years. Experts who drew on many ideas and updated readily (he called them foxes) forecast better than those who explained everything through one big theory (hedgehogs), and overall expert accuracy was modest [21BookExpert Political Judgment: How Good Is It? How Can We Know?Tetlock, 2005 · Princeton University Press].

Later, in a large forecasting tournament on 199 geopolitical questions, a roughly 45-minute training in basic probability improved forecasts across two years [22Primary studyPsychological strategies for winning a geopolitical forecasting tournamentMellers et al., 2014 · Psychological Science 25:1106-1115]. The top 2% of forecasters, placed into teams, kept outperforming everyone else the next year. These "superforecasters" were partly discovered and partly made: ability mattered, but so did training, teamwork and practice [23Primary studyIdentifying and cultivating superforecasters as a method of improving probabilistic predictionsMellers et al., 2015 · Perspectives on Psychological Science 10:267-281].

When patterns blind experts

The same speed that makes recognition useful can trap you. In the Einstellung effect, a familiar solution comes to mind so quickly that it blocks a better one. In chess problems with both a familiar, slower solution and a better, less obvious one, experts performed at about the level of players three standard deviations weaker [24Primary studyInflexibility of experts: reality or myth? Quantifying the Einstellung effect in chess mastersBilalić et al., 2008a · Cognitive Psychology 56:73-102]. Eye tracking showed that experts who said they were searching for a better move kept looking at the squares related to the familiar one [25Primary studyWhy good thoughts block better ones: the mechanism of the pernicious Einstellung (set) effectBilalić et al., 2008b · Cognition 108:652-661]. Stronger experts were less affected.

When pattern detection misfires

A system tuned to find structure fast will sometimes find structure that is not there.

  • Seeing faces in noise. Participants told that half of a set of pure-noise images contained faces reported seeing faces in 34% of them, with activity in the brain's face area [26Primary studySeeing Jesus in toast: neural and behavioral correlates of face pareidoliaLiu et al., 2014 · Cortex 53:60-77]. This is called pareidolia.
  • Illusory correlation. People see a link between a minority group and rare behaviours when both are simply uncommon [27Primary studyIllusory correlation in interpersonal perception: a cognitive basis of stereotypic judgmentsHamilton & Gifford, 1976 · Journal of Experimental Social Psychology 12:392-407], one way stereotypes form without any real basis.
  • A claim that did not survive. A 2008 study in Science reported that feeling out of control makes people see more illusory patterns [31Primary studyLacking control increases illusory pattern perceptionWhitson & Galinsky, 2008 · Science 322:115-117]. Seven later experiments found no such effect, along with signs of publication bias in the earlier literature [32Primary studyExperimental manipulations of personal control do not increase illusory pattern perceptionvan Elk & Lodder, 2018 · Collabra: Psychology 4:19].

The hot hand: a twist worth knowing

In 1985, a famous study asked basketball fans whether a player is more likely to score after making several shots in a row. Most said yes: 91% believed it [28Primary studyThe hot hand in basketball: on the misperception of random sequencesGilovich et al., 1985 · Cognitive Psychology 17:295-314]. The researchers then analysed real shooting records and found no evidence of streaks. Their conclusion, that the "hot hand" is an illusion of seeing patterns in randomness, became a textbook example of human bias.

Then, in 2018, economists Joshua Miller and Adam Sanjurjo found an error in the analysis itself [29Primary studySurprised by the hot hand fallacy? A truth in the law of small numbersMiller & Sanjurjo, 2018 · Econometrica 86:2019-2047]. In any finite sequence, the share of successes right after a streak is expected to be lower than the true rate, purely by chance. For a fair coin flipped 100 times, the expected share of heads right after three heads is about 46%, not 50% [30ReviewA bridge from Monty Hall to the hot hand: the principle of restricted choiceMiller & Sanjurjo, 2019 · Journal of Economic Perspectives 33:144-162]. Correcting for this bias turned the original study's small, non-significant difference into a significant hot-hand effect of about 13 percentage points.

Interactive lab

Find the hidden bias in a coin

Flip a fair coin a few times. Look only at the flips that come right after a streak of heads, and ask what share of them were heads. Average that share across many short sequences. A fair coin should give 50%. Does it?

40.5%Expected share of heads right after 1 head, averaged over sequences of 4 flips. A fair coin is 50%.
Six example sequences
THTT
0%
HTHT
0%
THHT
50%
THHT
50%
TTHH
100%
THTH
0%

Outlined flips are the ones that count.

Why it happens: in a short sequence, a head that follows a head uses up one of the few heads available, so the next counted flip is a little more likely to be tails. The bias shrinks as sequences get longer, but it never disappears for finite data. That is exactly the trap the original hot-hand studies fell into.

The expected value is computed exactly by listing every possible sequence (sequences with no flip after a streak are left out, as in the original studies).

Humans versus machines

Modern AI systems are powerful statistical pattern learners, and comparing them with people clarifies what human pattern recognition is good at [33ReviewBuilding machines that learn and think like peopleLake et al., 2017 · Behavioral and Brain Sciences 40:e253]. A deep learning system reached human-level play on 29 of 49 classic Atari games, but needed vastly more practice than people. Humans can learn a new handwritten character from a single example. Machines can also be fooled by tiny changes to an image that no person would notice [34Primary studyIntriguing properties of neural networksSzegedy et al., 2014 · International Conference on Learning Representations]. People, in turn, are fooled by randomness and noise that a well-calibrated statistical model would not be. Each has blind spots.

How to train your pattern recognition

Mix your examples (interleaving)

In a well-known study, people learned to recognise the styles of 12 painters. Some studied each painter's works in a block; others saw the painters mixed together [35Primary studyLearning concepts and categories: is spacing the "enemy of induction"?Kornell & Bjork, 2008 · Psychological Science 19:585-592]. When tested on new paintings, the mixed group was far more accurate. Yet most participants believed blocking had worked better.

Did mixing painters help people recognise new paintings?
Share of new paintings classified correctly, experiment 1a
Studied mixed (interleaved)61%
Studied in blocks35%
Source: Kornell & Bjork, 2008 [35Primary studyLearning concepts and categories: is spacing the "enemy of induction"?Kornell & Bjork, 2008 · Psychological Science 19:585-592]. 78% of participants did better with mixing, yet 78% rated blocking as equal or better.

Mixing forces you to notice what distinguishes one category from another, which is exactly what recognition needs. A meta-analysis of 59 papers confirmed the benefit, but showed it depends heavily on the material [36Meta-analysisSimilarity matters: a meta-analysis of interleaved learning and its moderatorsBrunmair & Richter, 2019 · Psychological Bulletin 145:1029-1052].

Does mixing help for every kind of material?
Effect of interleaving versus blocking (Hedges' g). Positive means mixing helped; negative means blocking was better.
Paintings+0.67
Overall+0.42
Photographs+0.35
Maths tasks+0.34
Word lists−0.39
Source: Brunmair & Richter, 2019 [36Meta-analysisSimilarity matters: a meta-analysis of interleaved learning and its moderatorsBrunmair & Richter, 2019 · Psychological Bulletin 145:1029-1052].

Mixing helps most when categories look similar to each other and when examples within a category vary a lot. For memorising simple word lists, blocking actually worked better.

Practise recognising, not just solving

Perceptual learning can be trained directly with many short "what is this?" trials and immediate feedback. In one algebra program, students practised recognising equation structure without solving anything. Their speed went from about 28 seconds per problem to about 12, and the gain held two weeks later [37Primary studyPerceptual learning modules in mathematics: enhancing students' pattern recognition, structure extraction, and fluencyKellman et al., 2010 · Topics in Cognitive Science 2:285-305]. Similar modules in dermatology produced large gains that lasted a year [38Primary studyTraining pattern recognition of skin lesion morphology, configuration, and distributionRimoin et al., 2015 · Journal of the American Academy of Dermatology 72:489-495]. These studies mostly lack randomized control groups, so the true effect may be smaller.

Your training plan, graded

  1. Check the environment before trusting your gutStrong evidence

    Ask two questions: are outcomes predictable from cues here, and have I had fast, honest feedback? If not, treat your confident feelings with suspicion [18ReviewConditions for intuitive expertise: a failure to disagreeKahneman & Klein, 2009 · American Psychologist 64:515-526, 19ReviewThe two settings of kind and wicked learning environmentsHogarth et al., 2015 · Current Directions in Psychological Science 24:379-385].

  2. Get lots of real examples from your own fieldStrong evidence

    Pattern libraries are domain-specific, even square-specific in chess [2Primary studyPerception in chessChase & Simon, 1973 · Cognitive Psychology 4:55-81, 4Primary studyRecall of random and distorted chess positions: implications for the theory of expertiseGobet & Simon, 1996b · Memory & Cognition 24:493-503]. General "brain training" will not build them.

  3. Mix similar-looking categories when you studyStrong evidence

    Mix diagnoses, problem types or styles rather than blocking them. Expect it to feel harder while it works better [35Primary studyLearning concepts and categories: is spacing the "enemy of induction"?Kornell & Bjork, 2008 · Psychological Science 19:585-592, 36Meta-analysisSimilarity matters: a meta-analysis of interleaved learning and its moderatorsBrunmair & Richter, 2019 · Psychological Bulletin 145:1029-1052]. Strong for visual categories, moderate for maths, not for simple word lists.

  4. Use a formula or checklist as a baseline in noisy fieldsStrong evidence

    Override it only when you know something it does not [20Meta-analysisClinical versus mechanical prediction: a meta-analysisGrove et al., 2000 · Psychological Assessment 12:19-30].

  5. Practise quick recognition with instant feedbackModerate evidence

    Many short classification trials, not only full problems [37Primary studyPerceptual learning modules in mathematics: enhancing students' pattern recognition, structure extraction, and fluencyKellman et al., 2010 · Topics in Cognitive Science 2:285-305, 38Primary studyTraining pattern recognition of skin lesion morphology, configuration, and distributionRimoin et al., 2015 · Journal of the American Academy of Dermatology 72:489-495].

  6. Create feedback where the world gives noneModerate evidence

    Keep a prediction log: write down forecasts with probabilities, then score them later. This turns a wicked environment a little kinder [22Primary studyPsychological strategies for winning a geopolitical forecasting tournamentMellers et al., 2014 · Psychological Science 25:1106-1115].

  7. Learn basic probabilityModerate evidence

    A short training improved forecasting for two years in a large field study [22Primary studyPsychological strategies for winning a geopolitical forecasting tournamentMellers et al., 2014 · Psychological Science 25:1106-1115].

  8. After your first idea, look for a better oneModerate evidence

    Experts are prone to Einstellung [24Primary studyInflexibility of experts: reality or myth? Quantifying the Einstellung effect in chess mastersBilalić et al., 2008a · Cognitive Psychology 56:73-102]. Deliberately look away from what triggered your first answer.

  9. Before believing a streak, ask what chance would produceStrong evidence

    Random data contain more streaks than people expect, and short samples are biased in subtle ways [28Primary studyThe hot hand in basketball: on the misperception of random sequencesGilovich et al., 1985 · Cognitive Psychology 17:295-314, 29Primary studySurprised by the hot hand fallacy? A truth in the law of small numbersMiller & Sanjurjo, 2018 · Econometrica 86:2019-2047].

  10. Treat first impressions as hypothesesModerate evidence

    Expert gist is real but weak [14Primary studyThe gist of the abnormal: above-chance medical decision making in the blink of an eyeEvans et al., 2013 · Psychonomic Bulletin & Review 20:1170-1175]. Use it to direct attention, then verify.

Myths and what the evidence says

Their memory advantage is mostly for chess as it is really played. On random boards it shrinks to about one extra piece per 400 rating points, versus about five on real positions [5ReviewRecall of rapidly presented random chess positions is a function of skillGobet & Simon, 1996c · Psychonomic Bulletin & Review 3:159-163].

De Groot found grandmasters did not search more or deeper than weaker players. They saw better candidate moves first [1BookThought and Choice in Chess (2nd ed.; Dutch original 1946)de Groot, 1978 · De Gruyter Mouton].

Intuition is best explained as recognition built from experience. It can be trained and it is tied to a domain [8Primary studyRapid decision making on the fire ground: the original study plus a postscriptKlein et al., 2010 · Journal of Cognitive Engineering and Decision Making 4:186-209, 18ReviewConditions for intuitive expertise: a failure to disagreeKahneman & Klein, 2009 · American Psychologist 64:515-526].

Confidence is not a reliable sign of accuracy. What matters is whether the environment is regular and whether you had valid feedback [18ReviewConditions for intuitive expertise: a failure to disagreeKahneman & Klein, 2009 · American Psychologist 64:515-526].

In 136 studies, formulas won or tied 94% of the time, and experience did not change that [20Meta-analysisClinical versus mechanical prediction: a meta-analysisGrove et al., 2000 · Psychological Assessment 12:19-30].

The original analysis contained a streak selection bias. Corrected, the same data show a hot hand [29Primary studySurprised by the hot hand fallacy? A truth in the law of small numbersMiller & Sanjurjo, 2018 · Econometrica 86:2019-2047, 30ReviewA bridge from Monty Hall to the hot hand: the principle of restricted choiceMiller & Sanjurjo, 2019 · Journal of Economic Perspectives 33:144-162]. People still misread randomness in general.

For visual categories, mixing works better. For simple word lists, blocking can win [36Meta-analysisSimilarity matters: a meta-analysis of interleaved learning and its moderatorsBrunmair & Richter, 2019 · Psychological Bulletin 145:1029-1052].

Most people did better with mixing, yet most judged blocking as equal or better [35Primary studyLearning concepts and categories: is spacing the "enemy of induction"?Kornell & Bjork, 2008 · Psychological Science 19:585-592].

The original 2008 finding [31Primary studyLacking control increases illusory pattern perceptionWhitson & Galinsky, 2008 · Science 322:115-117] did not replicate across seven experiments [32Primary studyExperimental manipulations of personal control do not increase illusory pattern perceptionvan Elk & Lodder, 2018 · Collabra: Psychology 4:19].

Machines can match people on benchmarks but need far more data, can be fooled by invisible changes, and lack causal understanding [33ReviewBuilding machines that learn and think like peopleLake et al., 2017 · Behavioral and Brain Sciences 40:e253, 34Primary studyIntriguing properties of neural networksSzegedy et al., 2014 · International Conference on Learning Representations].

Check your understanding

Question 1 of 5
Chess masters recall real positions far better than beginners. What happens with randomly scattered pieces?
Question 2 of 5
According to Kahneman and Klein, which two conditions make an intuition trustworthy?
Question 3 of 5
People who studied painters mixed together were more accurate on new paintings. What did most of them believe?
Question 4 of 5
Why was the original hot-hand study wrong?
Question 5 of 5
In noisy fields like clinical prediction, what does the evidence suggest?
Answered 0 of 5. Pick an answer to see the explanation.

Words to know

Pattern recognition
Matching what you perceive now to structures stored from past experience, to classify, predict or act.
Chunk
A group of elements stored and recalled as one unit, such as a familiar word or chess formation.
Template
A large, often-used chunk with a fixed core and open slots for the details of the current case.
Recognition-primed decision
Recognising a situation type, taking the action that comes with it, and checking it by imagining how it would play out.
Perceptual learning
Experience-driven improvement in noticing and extracting the information that matters.
Gist perception
Extracting the overall meaning of an image very quickly, before looking at details.
Statistical learning
Picking up regularities, such as which sounds follow which, from exposure alone.
Predictive processing
The theory that the brain predicts its input and updates its model when surprised.
Kind environment
A setting with quick, accurate feedback, where learning conditions match later use.
Wicked environment
A setting where feedback is delayed, missing or misleading, so experience can teach wrong lessons.
Interleaving
Mixing examples of different categories while studying, instead of grouping them.
Einstellung effect
A familiar solution coming to mind so fast that it blocks a better one.
Pareidolia
Seeing meaningful patterns, such as faces, in random noise.
Illusory correlation
Seeing a relationship between things that are not actually related.
Streak selection bias
In finite data, success right after a streak is expected to look lower than the true rate.
Calibration
How well your stated confidence matches how often you are actually right.

Sources

Every claim on this page links to one of these. Primary studies and meta-analyses carry the most weight; books and secondary sources are used for history and context.

  1. 1
    de Groot AD (1978). Thought and Choice in Chess (2nd ed.; Dutch original 1946). De Gruyter Mouton.
    Grandmasters did not search deeper than weaker players; they saw the position differently.
  2. 2
    Chase WG, Simon HA (1973). Perception in chess. Cognitive Psychology 4:55-81.
    Chunking and the real-versus-random recall difference (one player per skill level).
  3. 3
    Simon HA, Chase WG (1973). Skill in chess. American Scientist 61:394-403.
    Reviewgwern.net
    Recall gradient by skill; random boards recalled at about 3 to 4 pieces at every level.
  4. 4
    Gobet F, Simon HA (1996). Recall of random and distorted chess positions: implications for the theory of expertise. Memory & Cognition 24:493-503.
    Supports an estimate of about 50,000 chunks; chunks are tied to specific squares.
  5. 5
    Gobet F, Simon HA (1996). Recall of rapidly presented random chess positions is a function of skill. Psychonomic Bulletin & Review 3:159-163.
    A small expert edge remains on random boards: about 1 piece per 400 Elo vs 5 on real ones.
  6. 6
    Gobet F, Simon HA (1996). Templates in chess memory: a mechanism for recalling several boards. Cognitive Psychology 31:1-40.
    Template theory; one master recalled nine boards, up to 160 pieces.
  7. 7
    Klein GA, Calderwood R, Clinton-Cirocco A (1986). Rapid decision making on the fire ground. Proceedings of the Human Factors Society Annual Meeting 30:576-580.
    Introduces the recognition-primed decision model.
  8. 8
    Klein G, Calderwood R, Clinton-Cirocco A (2010). Rapid decision making on the fire ground: the original study plus a postscript. Journal of Cognitive Engineering and Decision Making 4:186-209.
    26 commanders, 156 decision points; over 80% recognitional.
  9. 9
    Simon HA (1992). What is an "explanation" of behavior?. Psychological Science 3:150-161.
    Usually cited for the idea that intuition is recognition.
  10. 10
    Saffran JR, Aslin RN, Newport EL (1996). Statistical learning by 8-month-old infants. Science 274:1926-1928.
    Infants found word boundaries from syllable statistics after 2 minutes.
  11. 11
    Black A, Bergmann C (2017). Quantifying infants' statistical word segmentation: a meta-analysis. Proceedings of the 39th Annual Conference of the Cognitive Science Society, 124-129.
    Meta-analysisescholarship.org
    The infant effect replicates, but it is small and fragile.
  12. 12
    Clark A (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences 36:181-204.
    The predictive processing framework and its open problems.
  13. 13
    Kellman PJ, Garrigan P (2009). Perceptual learning and human expertise. Physics of Life Reviews 6:53-84.
    Perceptual learning as a core of expertise that ordinary teaching leaves untrained.
  14. 14
    Evans KK, Georgian-Smith D, Tambouret R, Birdwell RL, Wolfe JM (2013). The gist of the abnormal: above-chance medical decision making in the blink of an eye. Psychonomic Bulletin & Review 20:1170-1175.
    Radiologists sense abnormality above chance in 250 ms but cannot locate it.
  15. 15
    Raat EM, Farr I, Wolfe JM, Evans KK (2021). Comparable prediction of breast cancer risk from a glimpse or a first impression of a mammogram. Cognitive Research: Principles and Implications 6:72.
    Looking longer did not strengthen the gist signal.
  16. 16
    Gauthier I, Tarr MJ, Anderson AW, Skudlarski P, Gore JC (1999). Activation of the middle fusiform "face area" increases with expertise in recognizing novel objects. Nature Neuroscience 2:568-573.
    Primary studydoi:10.1038/9224
    Training on novel "Greeble" objects recruits the face area.
  17. 17
    Gauthier I, Skudlarski P, Gore JC, Anderson AW (2000). Expertise for cars and birds recruits brain areas involved in face recognition. Nature Neuroscience 3:191-197.
    Primary studydoi:10.1038/72140
    Car and bird experts show expertise effects in the face area.
  18. 18
    Kahneman D, Klein G (2009). Conditions for intuitive expertise: a failure to disagree. American Psychologist 64:515-526.
    Intuition is trustworthy only with a regular environment and a chance to learn it.
  19. 19
    Hogarth RM, Lejarraga T, Soyer E (2015). The two settings of kind and wicked learning environments. Current Directions in Psychological Science 24:379-385.
    Defines kind and wicked learning environments.
  20. 20
    Grove WM, Zald DH, Lebow BS, Snitz BE, Nelson C (2000). Clinical versus mechanical prediction: a meta-analysis. Psychological Assessment 12:19-30.
    136 studies: formulas beat or tied human judges in 94% of them.
  21. 21
    Tetlock PE (2005). Expert Political Judgment: How Good Is It? How Can We Know?. Princeton University Press.
    Eclectic "foxes" forecast better than single-theory "hedgehogs".
  22. 22
    Mellers B, Ungar L, Baron J, et al. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science 25:1106-1115.
    A short probability training improved forecasts for two years; top forecasters kept winning.
  23. 23
    Mellers B, Stone E, Murray T, et al. (2015). Identifying and cultivating superforecasters as a method of improving probabilistic predictions. Perspectives on Psychological Science 10:267-281.
    Superforecasters are partly discovered and partly created.
  24. 24
    Bilalić M, McLeod P, Gobet F (2008). Inflexibility of experts: reality or myth? Quantifying the Einstellung effect in chess masters. Cognitive Psychology 56:73-102.
    A familiar solution cut experts to the level of players about 3 SD weaker.
  25. 25
    Bilalić M, McLeod P, Gobet F (2008). Why good thoughts block better ones: the mechanism of the pernicious Einstellung (set) effect. Cognition 108:652-661.
    Eye tracking: experts kept looking at the familiar solution.
  26. 26
    Liu J, Li J, Feng L, Li L, Tian J, Lee K (2014). Seeing Jesus in toast: neural and behavioral correlates of face pareidolia. Cortex 53:60-77.
    People saw faces in 34% of pure-noise images.
  27. 27
    Hamilton DL, Gifford RK (1976). Illusory correlation in interpersonal perception: a cognitive basis of stereotypic judgments. Journal of Experimental Social Psychology 12:392-407.
    People see a link between a minority group and rare behaviours that is not in the data.
  28. 28
    Gilovich T, Vallone R, Tversky A (1985). The hot hand in basketball: on the misperception of random sequences. Cognitive Psychology 17:295-314.
    Fans believe in streaks; the authors concluded the hot hand was an illusion.
  29. 29
    Miller JB, Sanjurjo A (2018). Surprised by the hot hand fallacy? A truth in the law of small numbers. Econometrica 86:2019-2047.
    Proves the streak selection bias that undermined the original analysis.
  30. 30
    Miller JB, Sanjurjo A (2019). A bridge from Monty Hall to the hot hand: the principle of restricted choice. Journal of Economic Perspectives 33:144-162.
    About 46% expected after three heads in 100 flips; corrected data show a hot hand.
  31. 31
    Whitson JA, Galinsky AD (2008). Lacking control increases illusory pattern perception. Science 322:115-117.
    Original claim that feeling out of control makes people see patterns.
  32. 32
    van Elk M, Lodder P (2018). Experimental manipulations of personal control do not increase illusory pattern perception. Collabra: Psychology 4:19.
    Seven experiments failed to replicate the 2008 result.
  33. 33
    Lake BM, Ullman TD, Tenenbaum JB, Gershman SJ (2017). Building machines that learn and think like people. Behavioral and Brain Sciences 40:e253.
    Machines need far more data than people and lack causal models.
  34. 34
    Szegedy C, Zaremba W, Sutskever I, et al. (2014). Intriguing properties of neural networks. International Conference on Learning Representations.
    Primary studyarxiv.org
    Tiny, invisible changes can make a neural network misclassify an image.
  35. 35
    Kornell N, Bjork RA (2008). Learning concepts and categories: is spacing the "enemy of induction"?. Psychological Science 19:585-592.
    Mixing painters raised accuracy from 35% to 61%, though learners preferred blocking.
  36. 36
    Brunmair M, Richter T (2019). Similarity matters: a meta-analysis of interleaved learning and its moderators. Psychological Bulletin 145:1029-1052.
    Interleaving helps overall (g = 0.42) but the effect depends on the material.
  37. 37
    Kellman PJ, Massey CM, Son JY (2010). Perceptual learning modules in mathematics: enhancing students' pattern recognition, structure extraction, and fluency. Topics in Cognitive Science 2:285-305.
    Recognition practice cut time per algebra problem from about 28 to about 12 seconds.
  38. 38
    Rimoin L, Altieri L, Craft N, Krasne S, Kellman PJ (2015). Training pattern recognition of skin lesion morphology, configuration, and distribution. Journal of the American Academy of Dermatology 72:489-495.
    Perceptual learning modules in dermatology, gains retained at 1 year.

Finished Pattern recognition? Mark it to track your progress in the sidebar. It is saved in this browser only.