Weak Data, Small Samples, and Politicized Conclusions on LGBT Discrimination

The measurement, analytic, and interpretive decision-making displayed in much (though certainly not all) of the LGBT discrimination and well-being literature is troubling, indicative of a lack of standards, poorly defined concepts, impressionistic conclusions derived from small numbers of interviews, the politicization of results, and the overall novelty of the field.

According to a purported deep dive into the social scientific literature, discrimination against LGBT Americans has yielded “a huge human toll.” That was the news greeting readers of the December 19 issue of the Washington Post. Since I was the principal author of the amicus brief that authors Nathaniel Frank and Kellan Baker feature (as a foil) in the first paragraph of their Post article, I figured I should read it carefully.

I did, and what appeared there isn’t new news. It’s the same weak data, small samples, and politicized conclusions to which we have been treated for years. Half of the six studies Frank and Baker discuss in the Post even fail to “prove” that patterns of discrimination widely, systematically, and profoundly harm LGBT Americans.

The pair report that they “spent two years conducting the largest known review of the peer-reviewed scholarship on the relationship between anti-LGBT discrimination and health harms,” but no such comprehensive document is evident online—only a brief overview of findings and a primer on their methods. They began by screening more than 11,000 peer-reviewed articles, a process that yields—in the end—300 articles probing the association between anti-LGBT discrimination and health and well-being.

I don’t blame them for limiting their analyses. The measurement, analytic, and interpretive decision-making displayed in much (though certainly not all) of this literature is troubling, indicative of a lack of standards, poorly defined concepts, impressionistic conclusions derived from small numbers of interviews, the politicization of results, and the overall novelty of the field. It was, after all, not many decades ago that the study of sexuality commenced. It’s been dogged by weakness the entire way. Alfred Kinsey was not just a pathfinder in sexual science; he was the first of many methodological offenders—plenty of whom have had a vested interest in the results of their own studies.

As I said in my amicus brief, that anti-gay discrimination can diminish psychological and physical health is widely acknowledged. But with society’s recent changes in norms and values, there is little evidence that chronic, repetitive, and intense discrimination based on sexual orientation remains a health issue. Moreover, the “minority stress” perspective privileged in such research opposes the idea that gays and lesbians should be seen merely as victims of social stress. They—like any other minority group—have long drawn strength from association and from establishing alternative structures and values, all of which temper the effect of discrimination. Indeed, the concept of resilience, or rebounding from adversity, has a rich history across the social sciences.

Frank and Baker, however, capitalize on the recent explosion of interest in transgender studies to extend the narrative of oppression. Sociologists call this “frame extension.” To merit continued attention to a population whose average income often well exceeds that of heterosexual Americans, the “frame” has been extended to encompass self-identified transgender persons, whose social and workplace experiences, not to mention incomes, appear far more challenging than those of gay and lesbian Americans.

Indeed, the Post’s discussion of such research begins with a study of the cortisol (stress hormone) levels of 65 transitioning (female-to-male) study participants, asserting that “encountering barriers in access to public restrooms predicted higher levels of stress.” But even here a reading of the study shows something different. There is no measure of “encountering barriers in access to public restrooms.” The barrier is internal—a measure of what respondents reported feeling when “using gender-specific public bathrooms.” In other words, they use stress to predict stress. No wonder it’s statistically significant—but barely, and only detectable first thing in the morning after respondents awoke. There were no differences here in cortisol levels the rest of the day. It’s an odd, tiny, and rather weak study to lead with. Meanwhile, they elected not to assess solid international studies, including one that documents how “trans women” (on estrogen) exhibit a doubled risk of stroke or deep vein thrombosis. Now that’s a health outcome. And it’s not the result of stigma.

Another study they discuss is rather old—about how Massachusetts health care use by sexual minority men varied in the 12 months after the state’s supreme court ruled in favor of same-sex marriage (back in November 2003). It reveals modestly lower subsequent medical and mental health care visits and costs. While I’m not privy to the data in order to reanalyze it, alternative explanations seem plentiful. The state’s economy fared slightly better in 2004 than it did in 2003, with state unemployment rolls dipping 12 percent. (After all, depression and anxiety are not always associated with issues of sexuality.) Moreover, the results include six months’ worth of data from the period during which same-sex marriage was on its way but not yet legal. And when you scrutinize the details, you see only tiny year-over-year improvement: non-partnered men, for example, reported two fewer mental health care visits (23 that year, on average, instead of 25 the year before, and $2,100 in mental health care costs instead of $2,400). In other words, there is statistical significance—yes—but not substantive significance. Why feature 16-year-old data when the times have changed?

A more recent study described in the Post “found suicide attempts by LGBT youth dropped by 7 percent in states that legalized same-sex marriage.” It’s noteworthy, but in an ironic sort of way. That’s because a recent follow-up scrutiny of this very study and its data revealed “little evidence that SSM laws have reduced suicide attempts among teen sexual minorities, nor have they decreased the likelihood of suicide planning, suicide ideation, or depression.” In other words, when other researchers examined the data, they didn’t come to the same conclusion at all. Instead, reanalyses actually found “some evidence that SSM legalization via judicial mandate is associated with worse mental health for these individuals. . . .”

This sounded familiar. Three years ago, I discovered an error in one such study that claimed that anti-gay attitudes contributed to a 12-year reduction in lifespan for sexual minorities. It wasn’t true. (The author eventually admitted the mistake.) But this time there is no obvious error. So why such distinctive interpretations of the very same data? Because conclusions about the data and what they tell us are sensitive to different measurement decisions and analytic strategies. This is the dirty secret about research on small populations: you can often find what you’re looking for if you construct your study and build your analytic models in particular ways. So earnestly do some in this domain want research to aid social and political causes that they have become blind to the ways in which measures, analytic decisions, and basic interpretations of data are fragile or skewed.

I am not disputing the reality that anti-LGBT discrimination exists and that its consequences can be felt and suffered, as some solid studies document. “Anti-gay prejudice,” I observed two years ago, “has been perpetuated throughout history. Outright violence obviously has an ill effect on individuals’ health and wellness.” This is beyond dispute, and I lament whenever I hear news of maltreatment. But Frank and Baker overreach, claiming that “policies or practices that deny a gay couple a wedding cake are not mere expressions of religious freedom—they inflict genuine psychological harm.” This is where we part ways.

Additionally, it’s easy to equate genuine discrimination with perceptions of the same. There is a difference, and better scholars recognize this “tendency to ‘go beyond’ the available data and to make sinister attributions regarding the intentions of others toward the self.” In Sweden, where anti-LGBT stigma and prejudice may be at a global low, this curious development has emerged. Differences in psychological distress between gay/lesbian and heterosexual respondents have disappeared over the past 10 years. Real victimization has declined dramatically. But “perceived discrimination” by gay/lesbian respondents has risen over that time (from 32 percent in 2005 to 37 percent in 2015). Expectations of being victimized are no different in 2015 than they were in 2005. Good times—that is, disappearing stigma—are apparently not good enough. This makes it more difficult to figure out whether or not, as Frank and Baker simply assert, “stigma and prejudice . . . are responsible for the disproportionate health harms LGBT people experience.”

Prejudice and stigma can be real, consequential, and harmful. As a rare skeptic in a very progressive and aggressively activist discipline—sociology—I know about social stigma. I feel it every day. Enduring such stigma, however, is the cost of doing business as a dissenter from the lucrative world of health research about sexual orientation and gender identity from which Frank and Baker draw their conclusions—a domain whose taxpayer funding has nearly doubled in the past three years, from $166 million in grants distributed by the National Institutes of Health in 2016 to an expected $308 million spent in 2019. (And that’s just public funding—in a Republican administration, no less.)

There is more to the story of differential health outcomes between LGBT and heterosexual populations than that portion that can be explained by prejudice and discrimination from strangers, employers, and colleagues. (By contrast, friends and family members’ attitudes tend to matter more.) So when you define stigma as simple opposition to same-sex marriage—which still characterizes nearly one in three Americans—it’s just scholarly irresponsibility.

Hence, to suggest somehow that, accommodating conscience and religious views on marriage and sexual expression opens the door to widespread discriminatory acts against LGBT persons, is to overreach, as well as to ignore public opinion and the ability of a free market economy to simultaneously accommodate diversity of thought and religious liberty. Americans—whether they are shopping for a cake or for a college—have long voted with their feet, their voices, and their wallets, and they remain free to do so. But as the Swedish study suggests, declining stigma and prejudice in America may not be enough to satisfy those who still feel unmeasurable acts of discrimination.

Keep up with the conversation! Subscribe to Public Discourse today.