Overspecialization can lead to collective tragedy even when every individual separately takes the most reasonable course of action.
Whether or not experience inevitably led to expertise, they agreed, depended entirely on the domain in question. Narrow experience made for better chess and poker players and firefighters, but not for better predictors of financial or political trends, or of how employees or patients would perform.
In wicked domains, the rules of the game are often unclear or incomplete, there may or may not be repetitive patterns and they may not be obvious, and feedback is often delayed, inaccurate, or both.
Our greatest strength is the exact opposite of narrow specialization. It is the ability to integrate broadly.
AI has made tremendous progress, but challenges remain. In a truly open-world problem devoid of rigid rules and reams of perfect historical data, AI has been disastrous. IBM’s Watson destroyed at Jeopardy! and was subsequently pitched as a revolution in cancer care, where it flopped so spectacularly that several AI experts told me they worried its reputation would taint AI research in health-related fields. As one oncologist put it, “The difference between winning at Jeopardy! and curing all cancer is that we know the answer to Jeopardy! questions.” With cancer, we’re still working on posing the right questions in the first place.
Tellingly, Marcus gave me this analogy for the current limits of expert machines: “AI systems are like savants.” They need stable structures and narrow worlds.
Psychologist Barry Schwartz demonstrated a similar, learned inflexibility among experienced practitioners when he gave college students a logic puzzle that involved hitting switches to turn light bulbs on and off in sequence, and that they could play over and over. It could be solved in seventy different ways, with a tiny money reward for each success. The students were not given any rules, and so had to proceed by trial and error.* If a student found a solution, they repeated it over and over to get more money, even if they had no idea why it worked. Later on, new students were added, and all were now asked to discover the general rule of all solutions. Incredibly, every student who was brand-new to the puzzle discovered the rule for all seventy solutions, while only one of the students who had been getting rewarded for a single solution did. The subtitle of Schwartz’s paper: “How Not to Teach People to Discover Rules”—that is, by providing rewards for repetitive short-term success with a narrow range of solutions.
In totality, the picture is in line with a classic research finding that is not specific to music: breadth of training predicts breadth of transfer. That is, the more contexts in which something is learned, the more the learner creates abstract models, and the less they rely on any particular example. Learners become better at applying their knowledge to a situation they’ve never seen before, which is the essence of creativity.
Repetition, it turned out, was less important than struggle.
Here is the bright side: over the past forty years, Americans have increasingly said in national surveys that current students are getting a worse education than they themselves did, and they have been wrong. Scores from the National Assessment of Educational Progress, “the nation’s report card,” have risen steadily since the 1970s. Unquestionably, students today have mastery of basic skills that is superior to students of the past. School has not gotten worse. The goals of education have just become loftier.
Exploration is not just a whimsical luxury of education; it is a central benefit.
Winston Churchill’s “never give in, never, never, never, never” is an oft-quoted trope. The end of the sentence is always left out: “except to convictions of honor and good sense.”
Do not be an engineer, he said, be a producer. “The producer knows that there’s such a thing as a semiconductor, but doesn’t need to know its inner workings. . . . That can be left to the experts.” He argued, “Everyone takes the approach of learning detailed, complex skills. If no one did this then there wouldn’t be people who shine as engineers. . . . Looking at me, from the engineer’s perspective, it’s like, ‘Look at this idiot,’ but once you’ve got a couple hit products under your belt, this word ‘idiot’ seems to slip away somewhere.”
Fortunately, it is possible, even today, even at the cutting edge, even in the most hyperspecialized specialties, to cultivate land where both birds and frogs can thrive.
Facing uncertain environments and wicked problems, breadth of experience is invaluable. Facing kind problems, narrow specialization can be remarkably efficient. The problem is that we often expect the hyperspecialist, because of their expertise in a narrow area, to magically be able to extend their skill to wicked problems. The results can be disastrous.
The Danish proverb that warns “It is difficult to make predictions, especially about the future,” was right.
God does not play dice with the universe, Einstein asserted, figuratively. Niels Bohr, his contemporary who illuminated the structure of atoms (using analogies to Saturn’s rings and the solar system), replied that Einstein should keep an open mind and not tell God how to run the universe.
He emphasized that there is a difference between the chain of command and the chain of communication, and that the difference represents a healthy cross-pressure.
Compare yourself to yourself yesterday, not to younger people who aren’t you. Everyone progresses at a different rate, so don’t let anyone else make you feel behind. You probably don’t even know where exactly you’re going, so feeling behind doesn’t help. Instead, as Herminia Ibarra suggested for the proactive pursuit of match quality, start planning experiments.
As Supreme Court justice Oliver Wendell Holmes wrote a century ago, of the free exchange of ideas, “It is an experiment, as all life is an experiment.”