Why is racism bad?

Recently a young man on Twitter ask me: Why is racism bad?

It might seem like a troll question, but he went on to explain what he meant. He wasn’t talking about hating people or restricting rights based on race. He wasn’t talking about using racial slurs or promoting racial segregation.

He was using the word “racist” in the driest, most emotionless sense possible: If we know that there are statistical differences between difference races, then–all other things being equal–doesn’t it make sense to use that information about race as part of your rational judgment process? Why would it be bad to simply use part of the data?

(This is how people usually frame racism when they want to argue for things like racial profiling: why not use race in deciding who to hire or who to screen for bombs at the airport if it could improve the probability of the desired outcome?)

The simplest answer is that it’s lazy. It’s plain old lazy, and as a consequence it is also sloppy. It finds a convenient solution rather than an optimal solution to the “probability maximization” problem.

And since the people who frame the issue of racism this way like to use statistical terms like “probability” I’d like to introduce another statistical term into the conversation: multicollinearity.

multicollinearity

Let’s take a look at three world maps: a map of prevalence of female genital mutilation, a map of percent of the population that identifies as Muslim, and a map of the “distribution of the races” created in 1920.

Female genital mutilation map

Muslim Population Map

Stoddard_race_map_1920

Now, can we get a sense of what factor might be responsible for driving a high prevalence of female genital mutilation?

Well, some people say Islamic culture is a primary driver. And it is true that there is a bunch of overlap in those two maps, although there are some misses: Pakistan is highly Muslim, and has no female genital mutilation. The same is true of Algeria. But when you consider both large number of Islamic regions that have female genital mutilation and the large number of non-Islamic regions that do not, the relationship is pretty strong. As statisticians would say: Islamic culture accounts for “much of the variance” in female genital mutilation.

Reza Aslan says that female genital mutilation is not an Islamic culture problem, but a central African problem. Looking at these maps, I think the region would be better described as “north-eastern African and Middle Eastern”, so for the sake of argument let’s use that. Regional culture, rather than religious culture, is also a good–but imperfect–predictor of variance.  Malaysia sticks out as an exception: it has a lot of female genital mutilation but is not in the region of north-eastern Africa and the Middle East.

So neither of these predictive variables–religious culture and regional culture–is perfect. Both have a lot of overlap, but not perfect overlap, with the phenomenon of female genital mutilation.  From a purely statistical perspective, either one is a “good predictor”; but because they overlap with each other so much, there is absolutely no way of knowing which of the two factors is really to blame… or if, in fact, it’s due to some unique combination of the two.

That’s multicollinearity.

Now you can also use race as a predictor: even with the fairly simplistic “skin tone only” conceptualization of race in the 1920 map presented above. “Brown People” ends up being a pretty good predictor of the same regions where female genital mutilation occur. It isn’t perfect, of course: it doesn’t account for Malayasia’s female genital mutilation, and incorrectly predicts female genital mutilation in India. But then again, none of these variables is perfect.

Just eyeballing the maps above, it looks like “Brown People” is a worse predictor than “Islamic Culture” … but the nice thing about “Brown People” is that it tells you how to feel about someone just by looking at them. So although using “Brown People” to explain who to blame isn’t actually giving you the most predictive power… it’s convenient.

It’s a good lazy person’s way out.

Now, let’s look at some of those “profiling” situations where people like to use statistics to try to justify “rational racism”.

What predicts good job performance? A shit-ton of things. Not only obvious and easily quantified things like success at prior jobs (i.e. “experience”) and academic achievements, but millions of minutia of data that you get from face-to-face interviews.

Anyone who has real-world experience interviewing people for jobs and hiring new employees knows that a 30 minute interview is a goldmine rich with information. How someone speaks or dresses tells you something about their upbringing, their self-awareness, the degree to which they understand norms in business culture, their level of attention to detail… and all of these things are predictive of future performance, and all of them also tend to be related to each other.

Multicollinearity.

When someone is trying to argue in favor of “rational racism”, the hypothetical scenario I always hear goes like this: “If you’re going to hire someone and you have two applicants who are in all other ways the same, shouldn’t you hire the white guy over the black guy (or the Asian guy over the white guy) because of known statistical differences in IQ between the races?”

When I hear that question, I know immediately that the speaker has never been a hiring manager, and has little or no experience interviewing people.

You never get two applicants who are “in all other ways the same.”  You always get such a wealth of information from behavior and speech during an interview, not to mention the on-paper differences in resumes, that if you feel like the only way you can discern between two candidates is their race then you are simply bad at your job.

If you can’t pick up on the millions of subtle cues during an interview that are strong predictors of IQ, experience, motivation and competence, so that you have to rely on race as the only information that you can use to distinguish between two people, then you are lazy, or incompetent, or both.

The same argument goes for profiling people getting on planes. We have extremely sophisticated technology, both for detecting bomb-making materials and for detecting psychological states such as nervousness from body language, that if you need to rely in race to figure out who to pull aside and check at the security line in the airport, then you are an incompetent screener.


Personally, I’m not so sure it’s a good idea to try to analyze “racism” in terms that are completely devoid of emotion or cultural and historical context. But I understand the desire. You don’t want the dialogue mired in obsessions with hurt feelings or abstractions like “structural oppression”.  So instead, you try to limit the conversation to just the question of rationality: let’s only look at the aspect of racism  that can be understood in terms of correlations, prediction and probability.

Maybe you want to embody the stalwart, emotionless, perfectly rational probability-calculating machine. That’s fine. But if you’re using race as your primary predictive variable of behavior in any of your day-to-day calculations, then–and I say this to you as emotionlessly and mathematically as possible: you’re doing it wrong.



6 views shared on this article. Join in...

  1. If it even were possible to find two applicants who are “in all other ways the same,” it wouldn’t just be lazy to choose the white guy over the black guy, it would actually be the wrong choice.

    By being one of two applicants who are “in all other ways the same,” the black guy has likely had to overcome so many more obstacles than the white guy and so is likely to be far more capable of achieving more, and hence choosing the black guy would be far more rational

    If the white guy had a higher IQ (laying aside arguments that IQ tests are culturally biased and also ignoring the fact that IQ is a poor predictor of performance) then wouldn’t he have achieved more? And hence there is no way that the two applicants could be “in all other ways the same.”

    • Greg Stevens says:

      This is a really good point, and I guess it does point back to problems with the fantasy scenario of the “all other things being equal”. If by “all other things being equal” one simply means the list of accomplishments or test scores or professional achievements, it’s certainly reasonable to assume that the black person with that list of achievements may have required some more motivation and perseverance than the white person to accomplish those same things. Of course, it may not have required such: depending on where each of them grew up and so on. But one certainly can’t take for granted that the same outcome implied the same underlying work-related personality traits.

      Excellent point, thanks for your comment.

      • Francis says:

        Really enjoyed this article, especially the interview conversation. I cannot even imagine a scenario where two candidates would be equal, since individuals have an infinite number of qualifiers, as well as deal-breakers. What if both candidates are of color? You still will find a better of the two, regardless of their genetics. As a manager I have hired a number of different ethnicities over whites because they were simply right for the job. As Greg points out, if you don’t hire the right person because of their qualities, you’re bad at your job.

        I am currently watching my company eliminate Americans from their positions and filling them with foreigners. The fear mongering is simply part of an agenda… who’s agenda is everybody’s guess. Hopefully people will go back to treating everybody as who they are instead of what we’re being told to think of them.

  2. DERP says:

    As I said on twitter, would like to see how you would argue this against Sam Harris talking about this subject
    http://www.samharris.org/blog/item/in-defense-of-profiling

    • Greg Stevens says:

      Hey buddy. When I asked you to leave your comment that you made on Twitter, I was actually referring to a different one: you argued that race should never be used as the only factor in judging a person, but if it is used in conjunction with other factors (like the behavioral stuff mentioned in my article) then it could lead to an even better results.

      The basic assumption sees to be an overall premise that “using more factors is better than using fewer factors” when trying to arrive at the best prediction.

      The thing is… that’s really not always true. In fact, when doing practical work as an analyst or data scientist, one of the most common things you find when trying to construct predictive models is that using some combinations of variables can completely knock out another variable–make it so that it has no predictive power at all–even though that other variable may have had substantial predictive power on its own. This is just another out come from multicollinearity: sometimes X is highly predictive of your result, but if you include A, B and C in the model instead, there is literally no additional gain from adding X.

      The way I describe it is that the variance accounted for by X has been “soaked up” by the other variables.

      Now, nobody (to my knowledge) has done this particular form of the experiment and analysis to say for sure, but I highly suspect that in certain situations — especially things like job interviews, and possibly also with screenings at airports — that if you choose the right set of diagnostic behavioral variables to try to predict who will do well at the job (or who is more likely to be carrying bad stuff on an airplane), they will push out any diagnostic ability “race” would have had, and leave it with no predictive power of its own.

      That’s a hypothesis, of course. It would be interesting to see if someone could test it empirically.

      The Sam Harris post you linked to seemed irrelevant and boring to me. He didn’t actually present a scientific or statistical argument. He just complained about how he hated being inconvenienced at airports. And then he assumed that adding racial profiling TO behavioral profiling would somehow provide better data. Personally, I’d like to see him address the question of multicollinearity in a serious way, and defend the assumption that racial profiling would be able to account for variance not already accounted for by properly profiling people on the basis of behavior or other aspects of physical appearance not related to race or religion (but related instead to things such as nervousness or the appearance of trying to conceal things).



Pings to this post

  1. […] baggage… the entire thing was a shit-show from beginning to end. There was way too much multicollinearity to ever point to a single “reason” that Trump […]