A recent study called “Shaping the Nation: Estimating the Impact of Fourth of July Using a Natural Experiment” came out of Harvard claiming that people who attended 4th of July celebrations as children are more likely to be Republicans when they grow up. From this conclusion, The Week took the reverse: Republicans are more patriotic than Democrats.

Now, I fully admit I didn’t go to Harvard, but I did look at the paper produced by the researchers and I noticed one major item of research missing.

They didn’t actually ask anyone if they attended any celebrations.

Their conclusion has absolutely nothing to do with celebrations or celebration attendance. Their conclusion is that people whose childhood had more “rain-free” July 4ths are more likely to be Republicans. So if it didn’t rain on the 4th of July a lot when you were a kid, you’re likely sizing up the GOP field to see who you’re going to vote for.

Sounds solid, right?

I don’t think so, either.

Please bear with me, this gets convoluted.

The researchers took rainfall data (I am not making this up) and used whether it rained in a given county on the 4th of July on average  (not, for example, in any given year) as a stand-in for actual attendance at a celebration.

These researchers also didn’t collect the political data; they used the data collected by an election polling group which the group makes available to the public, and took information from a variety of surveys, with a variety of data. Though they don’t explicitly say that they use many different surveys, each with its own margin of error, it is clear from how they “collected” the childhood data.

The thing is, the political data they used didn’t, for the most part, say where people had grown up. So for the group where the money conclusion was drawn–that children who attended 4th of July celebrations are more likely to be Republican–they used data only from people who indicated, in some
way (it’s fuzzy in their methodology, and I don’t actually see the surveys
used)  that they were still living in the region of their birth (it’s important to note that it’s the region of their birth; whether they still live in the same county, the unit used to determine attendance, does not seem to be included in the data. Also worth noting: immigrants are automatically excluded by this method).

So this is what we have so far: we have no actual data on whether the people in question ever attended a celebration. We have average rain fall. And we have nothing to show where people lived as children. They don’t define “region,” but if you take it as it commonly understood– midwest, south, west, northeast, etc–they make a huge leap. Even where people live in the same region, there’s nothing to prove they live in the same county. Chicago’s a long way from Cleveland, but both are in the same region.

If this study shows anything at all, which is questionable, it’s that there is a slightly higher chance of being a Republican if you still live in the region of your birth. Keep in mind, though, that the researchers found even this tenuous connection after controlling for factors like race, gender, and income.

What does that mean?

They basically took race, gender and income out of the equation, three of the individual characteristics most likely to have a relationship with choice of political party. So if you remove race, gender and income, people still living in the region of their birth may be slightly (3.6%) more likely to be Republicans.

Not exactly headline-grabbing.

Claiming that 4th of July rainfall is not only connected to choice of party, but causing it, is an abuse of statistics.

You can cloak it in big words (and trust me, there are many I had to look up) and dress it up in Harvard credentials, but bad science is bad science.