Alarm fatigue: tuning thresholds to avoid false alerts
How to set river and rainfall alert thresholds so that every alert is believed, using hysteresis, persistence, phased warnings and a plain count of false alarms.
A siren sounds at night because a river gauge touched its warning level for one minute, then fell back. It sounds again an hour later, and again the next evening. By the third week nobody in the control room looks up, and nobody in the village leaves the house. When the river finally does rise, the alert is correct and it is ignored. This is alarm fatigue: people and systems stop responding to an alert because too many earlier ones asked for action that was not needed. In flood warning, a high false alarm rate is one cause. Another is a threshold that is crossed too often, even when the crossings are real. This article explains how thresholds can be tuned to keep alerts rare enough to be believed and early enough to be useful.
What is a false alarm in flood warning?
A false alarm is a warning that is issued when the event it warned of does not happen. The false alarm rate (FAR) in the verification literature is the fraction of issued warnings that were not followed by the event [1]:
FAR = false alarms / (hits + false alarms)
PoD = hits / (hits + misses) (probability of detection)A hit is a warning followed by the event, a miss is an event that had no warning, and PoD is the share of events that were warned. The two measures pull against each other. Lowering the threshold catches more events and raises PoD, but it also raises FAR. Raising the threshold does the opposite.
Two points are easy to overlook. First, a false alarm is not always a mistake. WMO's Manual on Flood Forecasting and Warning says plainly that "false" alarms are inevitable given the nature of meteorological and hydrological events [2]. Second, FAR counts a near miss, a flood that stays 10 cm below the warned level, exactly as it counts a false trigger from a faulty sensor. Barnes and co-authors argued that this binary scoring gives forecasters no credit for close calls and proposed a model that separates them [1]. The lesson for tuning is to record why each alert did not verify, because the cure depends on the reason.
Does a false alarm make people ignore the next warning?
The evidence is mixed, and it is worth stating that honestly. The idea that too many false alarms make the public ignore later warnings, the "cry wolf" effect, is widely held. The review by Barnes and co-authors questions this conventional wisdom [1]. A survey study of tornado warnings in the United States by Trainor and co-authors found that the false alarm rate in a person's county did not predict how high that person thought the rate was, but it did predict behaviour: the actual false alarm rate in a person's county did predict behavioural response to tornado warnings [3]. The same study found that people define "false alarm" in very different ways, and that having a plan also mattered [3].
For floods specifically, a simulation study of a socio-hydrological model reproduced the cry wolf effect, with frequent false alarms eroding trust and willingness to act. It found that trust matters most where floods are infrequent, and that as forecasts improve, system performance becomes more sensitive to trust, so forecasters should set warning thresholds with social factors in mind [4]. That is a model, not field evidence.
How many alerts is too many?
WMO's manual gives indicative figures. Triggers "should be set so that they are not reached too frequently", because frequent triggering "can also lead to a negligent attitude by operators and public" [2]. Over time, false alarms should be a small proportion of the warnings issued, "for example 20 per cent". For places where flooding is irregular, no more than about five warning events per year, averaged over a period of years, is "considered to be of the correct order" [2]. These are indicative numbers, not targets for every river.
The manual also describes the balance: there is "a crucial balance to be struck between a precautionary approach and an unwillingness to issue warnings for fear of these being 'wrong'" [2].
Where do false alerts come from?
In a station-based system most false alerts fall into four groups.
- Bad data. A spike from a sensor, a drifting reading or a floating object under a radar beam can cross a threshold for one reading. WMO's manual recommends checking incoming data against the known range of the sensor, physical limits and limits on the rate of change, with thresholds specified for each item [2]. See data quality control for hydromet time series.
- Chatter. A real level that hovers around the threshold, rising and falling by a few centimetres with waves, releases or small gusts of inflow, crosses it many times without anything changing in the river.
- A threshold that is the wrong number. The manual warns that triggers "must not be arbitrary, or standard within an organization", for example within 1 metre of "danger" level, but must be tied to local features such as the level at which water leaves the channel or floods a low-lying road [2].
- Rain triggers that ignore wetness. Rainfall trigger values set low enough to cover the worst case, a saturated catchment, produce "an undesirably high number of false alarms" when the soil is dry, which is why the manual favours methods that account for catchment conditions [2].
How do hysteresis and persistence stop repeated alerts?
Two simple rules deal with chatter and spikes before any forecast is involved.
Hysteresis means using a different level to clear an alert than to raise it. The alert is raised when the level reaches the alert level, and is cleared only when it falls below a lower clear level. Between the two levels the state does not change.
Persistence means requiring the condition to hold for a set number of consecutive readings before the alert is raised. It removes one-reading spikes but costs time.
A worked example (illustrative)
Take a station that reads the water level every 5 minutes, with an alert level of 3.00 m. Twelve readings arrive:
Reading: 1 2 3 4 5 6 7 8 9 10 11 12
Level m: 2.96 3.01 2.98 3.02 3.03 2.99 3.04 3.05 3.07 3.10 3.12 3.15| Rule | Alerts raised | When the first alert is raised |
|---|---|---|
| Fixed level, alert whenever level is 3.00 m or higher | 3 (at readings 2, 4 and 7) | Reading 2, at minute 5 |
| Alert level 3.00 m with a clear level of 2.90 m | 1 | Reading 2, at minute 5 |
| Alert only after 3 consecutive readings at or above 3.00 m | 1 | Reading 9, at minute 40 |
The fixed level produces three alerts from one rise. Hysteresis gives one, with no delay. Persistence also gives one, but 35 minutes after the first crossing, which on a fast hill stream may be the whole warning time. That is why persistence should be short, often two or three readings, and why it suits slow rivers better than flashy ones. On a fast-rising stream, hysteresis alone is often the better choice, with data checks to catch spikes.
Should you use one threshold or several?
Several. WMO's manual says a phased-alert system "goes some way to overcoming" the false alarm problem, as long as the stages and their implications are clearly understood, and that it should allow a warning to be downgraded when conditions improve [2]. A typical ladder is a watch, a warning, a severe warning and a downgrade, each tied to an action. In India, river gauge sites already have two reference levels, the warning level and the danger level, and the Central Water Commission's flood bulletins classify sites by them: "above normal flood" at or above the warning level but below the danger level, and "severe flood" at or above the danger level but below the highest flood level [5]. Our article on warning level vs danger level explains how those two levels are meant to be used.
Stages let a low threshold work as an internal alert to watch-keepers without disturbing the public, while the loud, wide alert is reserved for a higher stage. WMO's manual gives two kinds of water-level trigger: the level rising to within a set distance of danger level, and the rate of rise exceeding a set value, with 25 cm per hour given as an example [2]. A rate trigger can raise the alert earlier on a fast rise without lowering the level threshold, so the level stays at its meaningful local value. The manual stresses that such triggers must be worked out for each site, with local knowledge [2].
How do you measure whether tuning is working?
Count, and keep the count. After each event or season, list every alert and mark it: hit, false alarm, or near miss, with the cause. Then compute FAR and PoD as above. An illustrative season:
| Before tuning | After raising the alert level | |
|---|---|---|
| Alerts issued | 10 | 6 |
| Followed by the event (hits) | 7 | 6 |
| Not followed by the event (false alarms) | 3 | 0 |
| Events with no alert (misses) | 1 | 2 |
| FAR = false alarms / alerts | 3 / 10 = 30 % | 0 / 6 = 0 % |
| PoD = hits / (hits + misses) | 7 / 8 = 88 % | 6 / 8 = 75 % |
The table shows the cost of tuning in one direction: removing the false alarms also lost one real event. The answer is rarely a single higher number. Check whether a rate-of-rise trigger would have caught the missed event, and whether a spike check would have removed the false alarms. WMO notes that for a high-impact event false alarms are more acceptable than misses, but that if the probability of occurrence is too low relative to the action required, the alert "will be ignored" [2]. As users gain experience, the manual says, triggers can be modified to reflect the balance of achieved forecasts and false alarms [2].
Beyond the threshold
Thresholds are one part of credibility. WMO's checklist for early warning systems lists "false alarms minimized and improvements communicated to maintain trust in the warning system" among its items [6]. Telling the community when a warning did not verify, and why, is part of that. Clear wording and instructions matter as much as the trigger; see writing warning messages people act on and India's colour-coded weather warnings.
Key points
- Alarm fatigue is a loss of response to alerts that came too often or too needlessly. FAR and PoD trade off against each other.
- The evidence on whether false alarms reduce later response is mixed, but a study of tornado warnings found that the actual false alarm rate in a county predicted how people responded [3].
- WMO's manual gives indicative figures: false alarms a small share of warnings (for example 20 %) and no more than about five warning events a year where floods are irregular [2].
- Clean the data first, then use hysteresis, short persistence, phased stages and a rate trigger, each tied to local features of the river.
- Keep a log of every alert, label it hit, false alarm or near miss with its cause, and review FAR and PoD each season.
Where ClimMaTech fits
At ClimMaTech we set up river and rainfall alerts as stages with separate alert and clear levels, so that a station sends one alert per rise rather than a stream of them. Rakshak Control Centre is available for demonstration, with live alerts requiring authorised approval, which keeps a human check between a threshold crossing and a siren. Our flood warning solution page describes how stations, thresholds and sirens fit together.
References
- R. B. Barnes, E. C. Gruntfest, M. H. Hayden, D. M. Schultz, C. Benight, False alarms and close calls: a conceptual model of warning accuracy, Weather and Forecasting 22(5), 1140–1147, 2007. doi:10.1175/WAF1031.1 (abstract opened via Crossref)
- World Meteorological Organization, Manual on Flood Forecasting and Warning, WMO-No. 1072, 2011, chapters 4, 6, 7 and 8. library PDF
- J. E. Trainor, D. Nagele, B. Philips, B. Scott, Tornadoes, social science, and the false alarm effect, Weather, Climate, and Society 7(4), 333–352, 2015. doi:10.1175/WCAS-D-14-00052.1
- Y. Sawada, R. Kanai, H. Kotani, Impact of cry wolf effects on social preparedness and the efficiency of flood early warning systems, Hydrology and Earth System Sciences 26(16), 4265–4278, 2022. doi:10.5194/hess-26-4265-2022
- Central Water Commission, Daily Flood Bulletin (Flood Forecast Monitoring Directorate), 6 May 2021. cwc.gov.in
- World Meteorological Organization, Multi-hazard Early Warning Systems: A Checklist, outcome of the Multi-hazard Early Warning Conference, Cancún, 22–23 May 2017, 2018. UNDRR Global Platform PDF