‘She’s never found peace since she left his arms, and never will again till she’s as he is now!’
—Thomas Hardy. Jude the Obscure. (1895).
“Done because we are too menny,” writes little “Father Time,” in Thomas Hardy’s Jude the Obscure—a suicide note that is meant to explain why the little boy has killed his siblings, and then hanged himself. The boy’s family, in other words, is poor, which is why Father Time’s father Jude (the titular obscurity) is never able, as he wished, to become the scholar he once dreamed of becoming. Yet, although Jude is a great tragedy, it is also something of a mathematical textbook: the principle taught by little Jude instructs not merely about why his father does not get into university, but perhaps also about just why, as Natasha Warikoo remarked in last week’s London Review of Books blog, “[o]ne third of Oxford colleges admitted no black British students in 2015.” Unfortunately, Warikoo never considers that possibility suggested by Jude: although Warikoo considers a number of reasons why black British students do not go to Oxford, she does not consider what we might call, in honor of Jude, the “Judean Principle”: that minorities simply cannot be proportionately represented everywhere always. Why? Well, because of the field goal percentages of the 1980-81 Philadelphia 76ers—and math.
“The Labour MP David Lammy,” wrote Warikoo, “believes that Oxford and Cambridge are engaging in social apartheid,” while “others have blamed the admissions system.” These explanations, Warikoo suggests, are incorrect: due to interviews with “15 undergraduates at Oxford who were born in the UK to immigrant parents, and 52 of their white peers born to British parents,” she believes that the reason for the “massive underrepresentation” of black British students is “related to a university culture that does not welcome them.” Or in other words, the problem is racism. But while it’s undoubtedly the case that many people, even today, are prejudiced, is prejudice really adequate to explain the case here?
Consider, after all, what it is that Warikoo is claiming—beginning with the idea of “massive underrepresentation.” As Walter Benn Michaels of the University of Illinois at Chicago has pointed out, the goal of many on the political “left” these days appears to be a “society in which white people were proportionately represented in the bottom quintile (and black people proportionately represented in the top quintile)”—in other words, a society in which every social strata contained precisely the same proportion of minority groups. In line with that notion, Warikoo assumes that, because Oxford and Cambridge do not contain the same proportion of black British people as the larger society does, that necessarily implies the racism of the system. But such an argument betrays an ignorance of how mathematics works—or more specifically, as MacArthur grant-winning psychologist Amos Tversky and his co-authors explained more than three decades ago, how basketball works.
In “The Hot Hand in Basketball: On the Misperception of Random Sequences,” Tversky and company investigated an entire season’s worth of shooting data from the NBA’s Philadelphia 76ers in order to discover whether there was evidence “that the performance of a player during a particular period is significantly better than expected on the basis of the player’s own record”—that is, whether players sometimes shot better (or “got hot”) than their overall shot record would predict. Prior to the research, it seems, everyone involved in basketball—fans, players, and coaches—appeared to believe that sometimes players did “get hot”—a belief that seems to predict that, sometimes, players have a better chance of making the second basket of a series than they did the first one:
Consider a professional basketball player who makes 50% of his shots. This player will occasionally hit four or more shots in a row. Such runs can properly be called streak shooting, however, only if their length or frequency exceeds what is expected on the basis of chance alone.
In other words, if a player really did get “hot,” or was “clutch,” then that fact would be reflected in the statistical record by a showing that sometimes players made second and third (and so on) baskets at a rate higher than that player’s chance of making a first basket: “the probability of a hit should be greater following a hit than following a miss.” If the “hot hand” existed, in other words, there should be evidence for it.
Unfortunately—or not—there was no such evidence, the investigators found: after analyzing the data for the nine players who took the vast majority of the 76ers shots for the 1980-81 season, Tversky and company found that “for eight of the nine players the probability of a hit is actually lower following a hit … than following a miss,” which is clearly “contrary to the hot-hand hypothesis.” (The exception is Daryl Dawkins, who played center—and was best known, as older fans may recall, for his backboard shattering dunks; i.e., a high-percentage shot.) There was no such thing as the “hot hand,” in short. (To use an odd turn of phrase with regards to the NBA.)
Yet, what has that to do with the fact that there were no black British students at one third of Oxford’s colleges in 2015? After all, not many British people play basketball, black or not. But as Tversky and his co-authors argue in “The Hot Hand,” the existence of the belief in a “hot hand” intimates that people’s “intuitive conception of randomness depart systematically from the laws of chance.” That is, when faced with a coin flip for example “people expect even short sequences of heads and tails to reflect the fairness of a coin and contain roughly 50% heads and 50% tails.” Yet, in reality, “the occurrence of, say, four heads in a row … is quite likely in a sequence of 20 tosses.” In just the same way, in other words, professional basketball players (who are obviously quite skilled at shooting baskets) are likely to make several baskets in a row—not because of any special quality of “heat” they possess, but instead simply because they are good shooters. It’s this inability to perceive randomness, in other words, that may help explain the absence of black British students at many Oxford colleges.
As we saw above, when Warikoo asserts that black students are “massively underrepresented” at Oxford colleges, what she means is that the proportion of black students at Oxford is not the same as the percentage of black people in the United Kingdom as a whole. But as “The Hot Hand” shows, to “expect [that] the essential characteristics of a chance process to be represented not only globally in the entire sequence, but also locally, in each of its parts” is irrational: in reality, a “locally representative sequence … deviates systematically from chance expectation.” Since Oxford colleges, after all, are much smaller population samples than the United Kingdom as a whole is, it would be absurd to believe that their populations could somehow exactly replicate precisely the same proportions as the larger population.
Maybe though you still don’t see why, which is why I’ll now call on some backup: professors of statistics Howard Wainer and Harris Zwerling. In 2006, the two observed that, during the 1990s, many became convinced that smaller schools were the solution to America’s “education crisis”—the Bill and Melinda Gates Foundation, they note, became so convinced of the fact that they spent $1.7 billion on it. That’s because “when one looks at high-performing schools, one is apt to see an unrepresentatively large proportion of smaller schools.” But while that may be so, the two say, in reality “seeing a greater than anticipated number of small schools” in the list of better schools “does not imply that being small means having a greater likelihood of being high performing.” The reason, they say, is precisely the same reason that you don’t have a higher risk of kidney cancer by living in the American South.
Why might you think that? Turns out, Wainer and Zwerling say, that U.S. counties with the highest apparent risk of kidney cancer are all “rural and located in the Midwest, the South, and the West.” So, should you avoid those parts of the country if you are afraid of kidney cancer? Not at all—because the U.S. counties with the lowest apparent risk of kidney cancer are all “rural and located in the Midwest, the South, and the West.” The county characteristics that tend to have both the highest and lowest rates of cancer are precisely the same.
What Wainer and Zwerling’s example shows is precisely the same as that shown by Tversky and company’s work on the field goal rates of the Philadelphia 76ers. It’s a “same” that can be expressed with the words of journalist Michael Lewis, who recently authored a book about Amos Tversky and his long-time research partner (and Nobel Prize-winner) Daniel Kahneman called The Undoing Project: A Friendship That Changed Our Minds: “the smaller the sample, the lower the likelihood that it would mirror the broader population.” If you flip a coin one hundred, or a thousand, times it is far more likely to come up heads half the time than if you flip it just ten or twenty times. Conversely, that small a sample is much more likely to come up biased towards either heads or tails—and much, much more likely to be heavily biased towards one or the other than a larger population of coin flips is. Getting extreme results is much more likely in smaller populations.
Oxford colleges are, of course, very small samples of the population of the United Kingdom, which is about 66 million people. Oxford University as a whole, on the other hand, contains about 23,000 students. There are 38 colleges (as well as some other institutions), and some of these—like All Souls, for example—do not even admit undergraduate students; those that that do consist largely of a few hundred students each. The question then that Natasha Warikoo ought to ask first about the admission of black British students to Oxford colleges is, “how likely is it that a sample of 300 would mirror a population of 66 million?” The answer, as the work of Tversky et al. demonstrates, is “not very”—it’s even less likely, in other words, than the likelihood of throwing exactly 2 heads and 2 tails when throwing a coin four times.
Does that mean that racism does not exist? No, certainly not. But Warikoo says that “[o]nly when Oxford and Cambridge succeed in including young Britons from all walks of life will they be what they say they are: world-class universities.” In fact, however, the idea that institutional populations ought to mirror the broader population is not only not easy to obtain—but flatly absurd. It isn’t that that a racially proportionate society is a difficult goal, in other words—it is that it is an impossible one. To get 300 people, or even 23,000, to reflect the broader population would require, essentially, rewiring the system to such an extent that it’s possible that no other goals—like, say, educating qualified students—could also be achieved; it would require so much effort fighting the entropy of chance that the cause would, eventually, absorb all possible resources. In other words, Oxford can either include “young Britons from all walks of life”—or it can be a world-class university. It can’t, however, be both; which is to say that Natasha Warikoo—like one character says about little “Father Time’s” stepmother, Sue, at the end of Jude the Obscure—will never find peace.