what is the optimal false alarm rate for tornado warnings?
Posted by Laura McLay on July 3, 2012
Last weekend, I patiently sat in my tiny downstairs bathroom with my three daughters and a laptop during two sequential tornado alarms. We followed the storm on twitter until it was safe to emerge. My husband didn’t fit in the bathroom, so he kept his eyes and ears ont he storm outside, occasionally complaining about high tornado warning false alarm rates.
My husband and I both have roots in the Midwest and are used to frequent tornado warnings. In Illinois, where sirens sounded during every tornado warning. Before I had kids, I usually ignored the alarms unless it looked or sounded ominous outside. I take shelter these days, at least when I know about the alarms. We don’t have tornado sirens in Virginia, so we often miss them. This makes me uneasy, since there are a lot of tornadoes in Virginia, even though Virginia isn’t in “tornado row.” But TV warnings and social media are usually sufficient for alerting me during warnings.
Tradeoffs.Tornado warning are supposed to be conservative, meaning that the false alarm rate is high in order to ensure a low false negative rate. The false negatives–the alarm not signaling before a tornado strikes–can be deadly. I’m OK with a lot of false alarms so long as I can keep my kids safe. There are social costs to have too many false alarms in order to drive the false negatives to zero. Namely, we will all have in our storm shelter during a perpetual tornado warning. There are social costs with having a reasonable number of tornado warnings. The main issue here is that false alarms are akin to “crying wolf”–tornado warnings are eventually ignored, leading to people being in danger when there really is a tornado.
It turns out that there is some research on this topic.
Error rates. First, I would like to note that there is about a 7% false negative rate. It’s not as low as I would like, but it’s certainly quite low considering the rarity of tornadoes. No system has perfect sensitivity and specificity. Now let’s discuss false alarms. The National Weather Service defines a false alarm ratio (FAR):
FAR = unverified warnings / (verified warnings + unverified warnings)
A paper by Kevin Simmons and Daniel Sutter in the Weather, Climate, and Society (an American Meteorological Society journal) studied areas with lower and higher false alarm rates. They found:
A statistically signiﬁcant and large false-alarm effect is found: tornadoes that occur in an area with a higher false-alarm ratio kill and injure more people, everything else being constant. The effect is consistent across false-alarm ratios deﬁned over different geographies and time intervals. A one-standard-deviation increase in the false-alarm ratio increases expected fatalities by between 12% and 29% and increases expected injuries by between 14% and 32%. The reduction in the national tornado false-alarm ratio over the period reduced fatalities by 4%–11% and injuries by 4%–13%. The casualty effects of false alarms and warning lead times are approximately equal in magnitude, suggesting that the National Weather Service could not reduce casualties by trading off a higher probability of detection for a higher false-alarm ratio, or vice versa.
The April 27, 2011 tornado outbreak in Alabama cost many lives and resulted in some discourse on the false alarm rate and what people are willing to live with. A meteorologist from Alabama argues for fewer tornado alarms:
I firmly believe apathy and complacency due to a high false alarm ratio over the years led to inaction in many cases that could have cost lives.
The FAR (false alarm ratio) for many NWS offices when it comes to tornado warnings is in the 80-90 percent category. I say this is simply not acceptable. Sure, the POD is excellent (probability of detection), but if most of the warnings are bad, then what good is a high POD?