I have come to realize that most people don’t understand what the word “all” means.
Here is one example. The following are two results from the same survey conducted by the Public Religion Research Institute on February 1-5, 2012. The margin of error is ± 3.5%.
“Please tell me if you completely agree, mostly agree, mostly disagree or completely disagree with [the following statement]: All employers should be required to provide their employees with health care plans that cover contraception or birth control at no cost.”
Completely or mostly agree: 55%
“There is currently a debate over what kinds of health care plans some religious organizations should be required to provide. Do you think religiously-affiliated colleges and hospitals should be required to provide their employees with health care plans that cover contraception or birth control at no cost, or not?”
Should be required: 49%
Some people might weasel over what it means to “mostly agree” with a statement, but taking this result at face value it means that 55% of the people think all employers should be required to cover contraception and 49% of people think that religious employers should be required to cover contraception.
This is logically and mathematically impossible. Any individual who believes that “all employers” should cover contraception must also believe that “religious employers” should cover contraception, because “religious employers” is a subset of “all employers.”
If you remember “Venn Diagrams” from high school or college, you’ll recognize the image on the left. If you think that “all employers should cover contraception” then you think that everything that falls inside the big circle should cover contraception. Since the entire smaller circle (religious employees) is inside the big circle, then you must also believe that all religious employers should cover contraception.
If 49% of people believe that “religious employers” should cover contraception and 55% of people believe that “all employers” should cover contraception, then there are 6% of people out there who think that ALL EMPLOYERS SHOULD cover contraception but RELIGIOUS EMPLOYERS SHOULD NOT cover contraception. It’s just not a belief that is logically possible to have.
Another example, from a CBS News/New York Times Poll conducted March 7-11, 2012. The margin of error is ±3%.
“Do you think health insurance plans for all employers should have to cover the full cost of birth control for their female employees, or should employers be allowed to opt out of covering that based on religious or moral objections?”
Should have to cover: 40%
“What about for religiously-affiliated employers, such as a hospital or university? Do you think their health insurance plans should have to cover the full costs of birth control for their female employees, or should they be allowed to opt out of covering that based on religious or moral objections?”
Should have to cover: 36%
Again, this is a matter of mathematics and logic. Every single person who believes that “all employers” should have to cover birth control absolutely must believe that religiously-affiliated employers should have to cover birth control. It’s a matter of logic: “religious employers” are a subset of “all employers.” It is logically impossible for someone to believe “all employers should have to cover birth control” and to not believe “religious employers should have to cover birth control.”
Therefore, it should be mathematically impossible for the percentage of people who believe “all employers should have to cover birth control” to be greater than the percentage of people who believe “religious employers should have to cover birth control.”
But there you have it. The actual survey results show something that is—or should be—logically impossible.
Why?
I think people don’t really understand what the word “all” means. More specifically, I think when people hear the word “all” on a survey, they don’t interpret it to mean “every”; instead, they interpret it to mean “average” or “regular” or “normal.”
Should the “average” employer have to cover birth control? Sure. Should religious employers have to cover birth control? Maybe not.
If I’m right about this, it is actually a very big deal in the world of survey data interpretation. Pollsters, pundits, politicians, and reporters are constantly debating, and often basing important decisions on, survey results by assuming that when people hear the word “all” the understand it to mean “every.” If that’s not true, there are dozens if not hundreds of surveys in just the last few years whose results have been, quite frankly, interpreted incorrectly.
Lots of people assume that inconsistencies in survey results come from deliberate partisan manipulation of wording. It seems to me that plenty of inconsistencies and misinterpretations come from mistakes that are much more innocent, but just as wrong.
Things like misunderstanding what the word “all” means.
