The hardest part of judging policies is the dreaded counter-factual. Basically, we want to know what would have happened if that policy didn't happen. Unfortunately, since most policies are not done in a lab, we rarely see a true counter-factual. The minimum wage raises or it doesn't. We only see one of the two.
Since we rarely have a counter-factual, what is someone with an interest in policy to do? Well, we do have some tools, but it can be rough.
Most of the fancy econometric tests are just trying to build this missing counter-factual. They attempted to reconstruct what would have happened under a different policy. If we have our best guess of what would have happened under a different policy (from the counter-factual), then we can start making some judgments about the relative outcomes.
As I said, this is really hard, I mean really hard, like perfect run in Mario hard.
We all struggle to find the right counter-factual. Sometimes we come quite short. In my Twitter feed, I came across a tweet and article which thinks it has the missing counter-factual.
— Economic Policy Inst (@EconomicPolicy) July 11, 2014
I'm not convinced this is the right counter-factual. Notice that I'm not disputing the "Myth." It might be a myth. I'm also not disputing the "Fact." I'm sure it's right.
I am doubting that the connection between the two sides of the picture are as clear as the Economic Policy Institute makes us believe.
The problem is that states with higher tipped minimum wage are likely completely different from lower tipped minimum wage states. Is it really fair to compare the richest states with the poorest and draw a conclusion? If I find that Haiti has a lower minimum wage than Sweden, should I conclude that raising Haiti's minimum wage to Sweden's would raise the pay of Haitians?
There are less drastic differences across states in the United States. But they still exist and we need to find a way to correct for those. That is where the econometrics can come in.
Simple facts like those that fit in 140 characters won't cut it. Sadly, most analysis shown in popular press won't cut it neither.
This is an easy trap to fall into when reading and writing. I'm not saying I know the right counter-factual often, but just giving a heads up to watch out for these slights of hand.