I wish critics would engage more with the meat of problematic ideas

Hello friends, I recently read a contemporary book of social science whose premise is so incendiary I’m afraid to reveal its name. My wife got mad at me for even reading the book, and she suggested that I should refrain from blogging about it. However, after reading the book, what disappointed me most was how shoddy its arguments were, especially given its sweeping conclusions. When I looked online for reviews of the book, I found no discussion of its actual methodology, I just found people attacking and discussing its conclusions. This seems to me a common problem. Oftentimes when you scratch the surface of controversial books, you find that the data really doesn’t support the conclusion–the book is simply not making a very convincing case. But everyone gets their back up about the conclusion and about the implications IF the conclusion happened to be true.

Thus, a lot of critiques of controversial ideas begin by implicitly adopting the premises of the author being critiqued. Maybe that’s because it’s simply easier and more fun to talk about big ideas rather than methods. But the problem is that truth matters–if a controversial idea is well-supported by data and argument, that’s very different from it being poorly supported. It’s like when people argue about treating trans kids and about whether it should be easier or harder for them to access care, a lot of people start by saying, “If you even question the idea that some kids shouldn’t get pills, then you’re supporting transphobic regimes in Texas and Florida!”

But there ought to be a step before that. And the step is, “Is there good data that kids are getting pills who shouldn’t?” And the answer seems to be…no. I haven’t seen any convincing argument in any of the various articles about trans kids that the current system, as it exists, is giving out pills too freely. It’s just a lot of _fear_ on the part of doctors that it _might_ be giving out pills too freely. Which is really lazy on their part. Do the research. This is an empirical question: how hard is it for kids to get pills? What assessments do gender clinics actually require? How much evidence of dysphoria do they actually need or desire? What is the status of gate-keeping as it currently exists?

Right now a lot of the research on this just seems profoundly lazy. It’s boring to actually create the methodologies, write the grants, make the connections, do the robustness checks you’d need to prove your assertions, and a lot of researchers simply don’t do it. They shoot from the hip instead. Or they come up with one little test or one little survey and extrapolate from that.

But when it comes to other people writing about the subject, it’s their responsibility to critique the evidence. Is this actually convincing? Has the researcher done enough work to make this claim? Researchers are often unwilling to criticize each other in print, so it can be up to the writer or journalist to do some digging themselves and to ask probing questions.

The risk in all these facile reactions is that you give readers the impression that there’s some truth here that you’re unwilling to engage. If trans kids were getting pills too easily and regretting it at higher rates than in the past, that would be really bad and worth addressing. But I saw nothing in, for instance, Emily Bazelon’s article, that led me to believe that this was actually the case. Nor did I gain any clear understanding of what the current system is for dispensing pills. All I saw was a lot of teens who navigated our complex health and mental health care systems and figured out the gender expression they needed / wanted and largely came to a bunch of conclusions that worked for them. There was certainly a lot of pain and soul searching involved, but there wasn’t strong evidence that kids are getting given pills like candy or that they’re undergoing hormone therapy without thinking it through. There was neither smoke nor fire, just a lot of anxiety that there might potentially be some smoke and someone could perhaps think about looking into it.

This is often the case with trend pieces though. They’re almost always false. A trend is a statistical phenomenon, and it needs to be supported with statistical analysis. Without that, you don’t have a trend, you’ve just got a bunch of anecdotes.

Anyway, the book I read was The Son Also Rises by Gregory Clark, a professor of economics at UC-Davis. It’s about social mobility across various societies. He uses one technique and one technique only, which is to look through historical registers of names, find rare surnames that are associated with elite status, and then look at contemporary registers of names and see how often those rare surnames show up in areas associated with elite status. For instance, he takes the names of Swedish nobility from the 17th century and sees how often they show up in registers of physicians, relative to surnames that are not especially associated with noble status. He has a lot of caveats and a lot of different ways of using the technique, but that’s it.

His main finding is that diffusion of these names isn’t nearly as rapid as you would expect. Names that were associated with high status in the 17th century are still disproportionately associated with high status in the 21st century. This conflicts with other findings about inheritance of wealth and education, which shows that families ought to regress to the mean after about three to five generations. Instead he says full regression should take about ten to fifteen generations. What he’s saying, essentially, is that because previous studies of social mobility took place over short timeframes (they often compare parents to kids, rather than comparing across five or six generations like he does), they overestimate the amount of randomness. But over enough generations and enough families, the randomness cancels out, because in reality a lot of the error terms are correlated with each other. It’s statistically complicated, and I have no desire to go into it here, but essentially he’s saying, look, if your grandparents were rich, you’re likely to be rich, even if your parents were not rich. He’s saying a family has a ‘natural’ social level, and sometimes through randomness they get below that level, but each generation they have a chance of regressing ‘back’ to their natural level. So impoverished nobility are more likely to get rich than impoverished middle-class people are.

This is a really cool study. If he’d just gone this far, it would’ve been neat: he’s demonstrated that class exists! Even in Sweden! In fact, he’s demonstrated the persistence of class across many different societies. I think that’s cool.

But Clark doesn’t mention class in the book. Nor does he mention racism or systemic bias. Instead, he can envision only two possible explanations for persistence of social status. Either your parentage must convey some material advantages, like money or education, or they must convey some genetic advantages.


You see where this is going? 

Clark leans heavily into the idea that what we’re seeing here is genetic superiority. That the Swedish nobles were on top in the 17th century because of their inherent fitness to rule, and that now, as a result, their umpteen-times grandkids have an inherent fitness to…become doctors? It doesn’t make a whole lot of sense.

Moreover, if that was a claim he wanted to prove, he could’ve done it in a number of ways. For instance, he makes a great thing about how even a thousand years later, Norman surnames are more likely to show up in Britain’s upper-classes. But if the elite in any given society is ruling because of inherent superiority, then shouldn’t the Anglo-Saxon elite have arisen again shortly after the invasion? What we ought to see is that in a few generations, Anglo Saxon names ought to have bounced back into the top ranks of society. But he doesn’t show this.

Indeed, he doesn’t seem particularly interested in really looking at the implications of his genetic inheritance idea and building a solid case that it’s true. Nor is he particularly interested in doing even the most basic robustness checks on his surname technique. Like, if you’re going to base an entire book on a single technique, at some point you need to figure out a basic question: “Am I learning something here about groups of people? Or am I just learning something about surnames?” Ideally what you’d do is compare your surname data’s results to some other results that you trust completely, to make sure that you’re not just learning something about how rare surnames diffuse and are adopted. But he never interrogates his own tool at all. He never introduces other sources of data either to calibrate his tool or check his results. Instead it’s all surname data.

It’s just profoundly lazy and quite disappointing, as is the fact that he doesn’t even think about addressing the idea that this lack of social mobility could be the result of racial or class bias. Instead, he treats as obvious the idea that, somehow, in some fashion, medieval society was a meritocracy, which seems inherently absurd. Like, why would William the Bastard’s ten thousand Norman followers have been inherently superior to the entire rest of the population of England? What would’ve been the mechanism for that? It doesn’t make any sense, and it goes profoundly against our intuitions.

But when I looked online for reviews of the book, I just found a lot of people calling it out for being racist (which it is), but none of the writers took the trouble to address the profound deficiencies in the argument, which is really disappointing to me. Indeed, the only reason I’m writing this blog post is so that if some future person googles “son also rises” “gregory clark” they’ll have this blog post as a referent.

I’m putting this subscription dingus at the bottom of every post now. Subscribe if you want to get my posts in your email, though I have no idea why you would willingly sign up for more spam.