Proofs
How accurate is it? Here’s every result, including the bad ones.
Every number on this site comes from this page. Each one is out of sample: a storm is only ever scored by a model trained on seasons before it, and checked against the fires agencies later recorded as lightning-caused.
01 · Australian back-test · AfterStrike v7
Most lightning fires start in a tenth of the cells.
Trained on New South Wales, Tasmanian, Victorian and Western Australian fire records. Tested on 2,423 agency-confirmed lightning fires, each counted once: 1,426 in NSW, 819 in Western Australia, 153 in Tasmania and 25 in Victoria. Every method gets the same number of flags. v7 also reads the new satellite hotspots seen overnight, before the 9am decision, so part of what it catches is fires already starting to show.
New South Wales · 1,426 fires · fire seasons 2015–16 to 2022–23
View as table
| Method | Top 10% of cells | Top 20% of cells |
|---|---|---|
| Lightning density only | 33% | 50% |
| Dry-lightning rule (today) | 46% | 67% |
| AfterStrike | 68% | 82% |
Tasmania · 153 fires · fire seasons 2015–16 to 2024–25
View as table
| Method | Top 10% of cells | Top 20% of cells |
|---|---|---|
| Lightning density only | 29% | 54% |
| Dry-lightning rule (today) | 54% | 73% |
| AfterStrike | 72% | 89% |
Western Australia · 819 fires · fire seasons 2015–16 to 2022–23
View as table
| Method | Top 10% of cells | Top 20% of cells |
|---|---|---|
| Lightning density only | 22% | 40% |
| Dry-lightning rule (today) | 29% | 44% |
| AfterStrike | 68% | 79% |
How sure are we? We resampled the storm days thousands of times and re-scored every method on the same fires each time.
Against the rule, AfterStrike came out ahead in 100% of resamples in New South Wales, 100% in Western Australia and 100% in Tasmania, though Tasmania’s smaller fire count leaves its range wide.
Against v6 (the same two models without the overnight hotspots), v7 came out ahead in 100% of resamples in NSW and 100% in Western Australia. In Tasmania it is level with v6 (72% against 73%).
Fires caught in the top 10% of cells, with 95% bootstrap range
| Method | Estimate | 95% low | 95% high |
|---|---|---|---|
| NSW · dry-lightning rule | 46% | 42% | 50% |
| NSW · AfterStrike v7 | 68% | 63% | 72% |
| Tasmania · dry-lightning rule | 54% | 32% | 69% |
| Tasmania · AfterStrike v7 | 72% | 58% | 82% |
| WA · dry-lightning rule | 29% | 21% | 37% |
| WA · AfterStrike v7 | 68% | 60% | 76% |
What made it better
- v0Tasmanian records onlyTasmania 62%*
- + NSW records3,000 more lightning fires from the NSW fire historyNSW 63% · Tasmania 62%
- + landscape, history, stormVegetation and terrain, earlier lightning fires, the storm around each cell (250 km), a 5-model ensembleNSW 63% · Tasmania 73%
- v5Plus fuel moisture, wind, soil moisture, terrain shape, greenness, people and climate driversNSW 61% · Tasmania 68%
- + WA recordsAbout 1,900 lightning fires from Western Australia's fire historyNSW 63% · Tasmania 66% · WA 56%
- v6A second model, a neural network that reads the storm map around each cell, averaged with v5NSW 64% · Tasmania 73% · WA 56%
- v7Both models also read the new satellite hotspots seen overnight, before the 9am decisionNSW 68% · Tasmania 72% · WA 68%
Top-10% catch per distinct fire, forward-chained (*v0 counted each fire once per storm it followed). Every step was tested on the same seasons, so differences of a few points between neighbouring steps are within noise. v6 and v7 were each confirmed across eight independent training runs of the map model. Training for longer, a wider map and softer labels were also tried and gave nothing.
02 · Forecast + satellite
Satellites catch the fires that grow.
The 9am flags tell crews where to look. Fire satellites then watch: a new hotspot inside a flagged cell is an alert. We replayed this on every storm since 2015–16 with the NASA MODIS and VIIRS hotspot archive (four to six passes a day). A fire counts if it was flagged at 9am, or seen by satellite within 4 km of where it started. “Forecast alone” is v7, which already reads the overnight passes, so the satellites add most where fires take a day or more to show.
| Forecast alone | + satellite within 48 h | Fires that reached 100 ha: forecast alone | + satellite, 48 h | + satellite, 5 days | |
|---|---|---|---|---|---|
| New South Wales · 1,426 fires | 68% | 73% | 68% of 451 | 78% | 87% |
| Tasmania · 153 fires | 72% | 75% | 77% of 26 | 88% | 92% |
| Western Australia · 819 fires | 68% | 78% | 53% of 356 | 70% | 79% |
The forecast makes hotspots worth chasing. Most new hotspots are burn-offs and ongoing fires. A new hotspot inside a cell AfterStrike flagged was a recorded lightning fire 28% of the time in NSW and 55% in Tasmania. Outside the storm’s cells, under 1%.
Small fires stay invisible. Satellites saw only 8% of NSW lightning fires that stayed under 10 ha within two days. For those, the 9am flag is the only warning, which is why the total stops short of 90%.
Western Australia’s big fires are the weak spot. Only 53% of WA lightning fires that reached 100 ha were flagged at 9am. Finding out why is next.
New hotspots that were lightning fires
- In a flagged cell28%
1,230 new hotspots
- Struck, not flagged3%
5,512 new hotspots
- Outside the storm0.6%
10,054 new hotspots
New South Wales, 2015–16 to 2022–23. A new hotspot: none within about 15 km in the week before.
03 · North America back-test
The same method, at continental scale.
Trained on US and Canadian fire seasons 2010–2017 and tested on the years after, against 68,336 lightning fires recorded by US agencies (FPA-FOD) and the Canadian National Fire Database.
United States + Canada, 2018–2020 · 50,293 fires
View as table
| Method | Top 10% of cells | Top 20% of cells |
|---|---|---|
| Lightning density only | 5% | 15% |
| Dry-lightning rule | 22% | 42% |
| AfterStrike | 62% | 77% |
Canada, 2021–2025 · 18,043 fires
View as table
| Method | Top 10% of cells | Top 20% of cells |
|---|---|---|
| Lightning density only | 13% | 25% |
| Dry-lightning rule | 21% | 37% |
| AfterStrike | 43% | 62% |
North American results use global weather (NASA POWER) and the same features as Australia. A North American model does not transfer to Tasmania: trained only there, it finds 15% of Tasmanian lightning fires in its top 10%, against 29% for the rule. Blending the two continents helped only marginally. Each country needs its own records.
04 · Storm replays
The busiest storm nights, flag by flag.
The storm days with the most confirmed lightning fires in each state, scored by AfterStrike v7 trained only on earlier seasons. Each method gets the same number of flags (20% of struck cells). Switch between them, and hover a cell to see why it scored as it did. Diamonds are the fires that followed within five days.
New South Wales · Black Summer, November 2019
New South Wales · 23 Nov 2019
Model trained on seasons up to 2018–19
78
Struck cells
16
Flagged
32
Fires caught (of 64)
New South Wales · 25 Nov 2019
Model trained on seasons up to 2018–19
155
Struck cells
31
Flagged
38
Fires caught (of 63)
New South Wales · 26 Nov 2019
Model trained on seasons up to 2018–19
125
Struck cells
25
Flagged
29
Fires caught (of 55)
Tasmania
Tasmania · 13 Jan 2016
Model trained on seasons up to 2014–15
25
Struck cells
5
Flagged
8
Fires caught (of 20)
Tasmania · 15 Jan 2019
Model trained on seasons up to 2017–18
26
Struck cells
6
Flagged
13
Fires caught (of 22)
Tasmania · 12 Mar 2025
Model trained on seasons up to 2023–24
39
Struck cells
8
Flagged
9
Fires caught (of 31)
Victoria (one labelled season)
Victoria · 2 Feb 2025
Model trained on seasons up to 2023–24
74
Struck cells
15
Flagged
2
Fires caught (of 10)
Victoria · 3 Feb 2025
Model trained on seasons up to 2023–24
52
Struck cells
11
Flagged
2
Fires caught (of 5)
Victoria · 4 Feb 2025
Model trained on seasons up to 2023–24
81
Struck cells
17
Flagged
3
Fires caught (of 4)
Mixed, as real results are. On the three worst Black Summer days in NSW, AfterStrike’s flags held 32, 38, 29 lightning fires against the rule’s 12, 14, 22. It lost one night of 9 to the rule (TAS 12 Mar 2025). In Victoria, with few fires on any night, it found 7 of 19.
05 · Where it fails today
What doesn’t work yet, and what fixes it.
Victoria, 2024–25 season (25 lightning fires)
View as table
| Method | Top 10% of cells | Top 20% of cells |
|---|---|---|
| Lightning density only | 40% | 52% |
| Dry-lightning rule (today) | 28% | 40% |
| AfterStrike | 48% | 64% |
It needs local records.
NSW records turned NSW from untested to the strongest result, and Western Australia's took WA to 56% against the rule's 29%. Victoria has one labelled season in open data, while its agencies hold thousands more lightning ignitions. That's the data partnership.
Satellite fire detections can't replace agency records.
Trained on 'new fires seen from space after lightning', the model learned where people burn off and did no better than chance.
Better lightning data didn't help.
We tested satellite lightning that locates every flash (GOES-16 GLM) on 24,000 US lightning fires. At 50 km and at 10 km it did no better than the free daily counts. Strike-level data is still needed to run live, but not for accuracy.
More of the same data has stopped helping.
In NSW, training on half the storm days scores the same as training on all of them; North American fires, smarter area searches and longer training added nothing. The gains left are better records (exact ignition points, Victoria, Queensland, South Australia) and satellites watching behind the forecast.
06 · How we test
The rules every proof follows.
- 01
Forward in time only
A season is scored by a model trained on earlier seasons. No random splits: storms are correlated, and mixing them inflates every score.
- 02
Agency records are the truth
A cell counts as a hit if a fire that a fire or parks agency recorded as lightning-caused starts in it within five days of the lightning.
- 03
Same number of flags
Every method flags the same share of struck cells, so the comparison is about ranking, not about who flags more.
- 04
Decision at 9am
Features use only what is known the morning after the storm: the lightning, the night's rain gauges, and fuel dryness from before the storm.
07 · Proof log
Every result, as it lands.
- Passed
Plantation country: 58–71% of lightning fires in the top tenth, ahead of the rule in every state
Re-scoring the v7 back-test where Australia's commercial plantations are (ABARES Forests of Australia 2023): ranking only the struck areas in 50 km cells holding at least 1,000 ha of plantation, and flagging the top 10% of them, catches 65% of the lightning fires there in NSW (385 fires; dry-lightning rule 52%), 71% in Tasmania (112; rule 54%) and 58% in Western Australia (414; rule 38%). AfterStrike is ahead of the rule in every resample in all three. Flagging by the whole state's top 10% instead looks better in WA (91%), but only because it flags over a third of the plantation country there, so we report the local figure. Victoria has too few recorded lightning fires near plantations (13) to say.
- Passed
v7: reading last night's satellite passes lifts WA from 56% to 68%
Both models now also read the new fire hotspots seen between the storm and the 9am decision (VIIRS and MODIS overnight passes). Per fire, the top 10% of struck cells catch 68% of NSW's lightning fires (v6 64%, ahead in every resample), 68% of Western Australia's (v6 56%) and 72% of Tasmania's (v6 73%, level). For fires that grew past 100 ha: NSW 68% (v6 59%), WA 53% (v6 29%). Part of the gain is fires already starting to show by morning, so in NSW the share of caught fires found after the flag drops from 79% to 74%. A second model trained only on fires that grew past 100 ha, and new hotspots from the two days before the storm, were tested and not adopted.
- Passed
Forecast + satellite (v6): 92% of Tasmania's lightning fires that grew past 100 ha
Replaying every storm since 2015–16 with the NASA MODIS and VIIRS hotspot archive behind the 9am flags: counting a fire if it was flagged, or seen by satellite within 48 hours, catches 73% of NSW's lightning fires (forecast alone 64%), 77% of Tasmania's (73%) and 72% of WA's (56%). For fires that reached 100 ha: Tasmania 92%, NSW 76% (86% within five days), WA 57%. A new hotspot inside a flagged cell was a recorded lightning fire 31% of the time in NSW and 59% in Tasmania, against under 1% outside the storm.
- Didn't work
Strike-level lightning, more history, North American data, area search: no gain
Satellite lightning that locates every flash (GOES-16 GLM, 24,000 US fires) scored no better than the free daily counts at 50 km or 10 km. In NSW the model scores the same trained on half the storm days as on all of them. Pre-training on, or mixing in, North American fires didn't help, nor did smoothing scores or searching around flagged cells at equal area. A quick Himawari fire detector we built was no faster than the polar satellites on the 25 Nov 2019 storm. The remaining gains are in better records and in satellites watching behind the forecast.
- Didn't work
Satellite storm rain (Himawari, every 10 minutes): no gain
Himawari's rainfall rate shows whether the storm actually rained where it struck, at 2 km. Added to the model alongside the gauge-based rain it reads now, it changed nothing: NSW +0 points, WA −2 (within noise), tested on the 2019–20 and 2020–21 seasons. Rain at the strike seems to be captured well enough by the gauges and the storm around each cell.
- Passed
v6: Western Australia added, and a second model that reads the storm map
About 1,900 Western Australian lightning fires from the national bushfire history became training labels, and a neural network that reads the weather and lightning maps around each struck cell now runs alongside v5, the two averaged. Per fire, the top 10% of struck cells catch 64% of NSW's lightning fires (v5 61%), 72.5% of Tasmania's (68%) and 56% of WA's (dry-lightning rule 29%). Against the rule it wins in every resample in all three states; against v5 alone the step is smaller (ahead in 89% of resamples in NSW, 97% in Tasmania). At 10 km, crews would search about a fifth less ground to find half the fires.
- Mixed
v5 updated: 16 more inputs, accuracy about the same
v5 now also reads live fuel moisture and flammability (ANU), wind, soil moisture, terrain shape, vegetation greenness, population and night lights, and climate drivers (ENSO, IOD, SAM), all known on the morning. Per fire, the top 10% of cells catch 61% in NSW (was 63%) and 69% in Tasmania (was 75%): within the uncertainty. Victoria improved from 15% to 27% (41 fires). Kept because the inputs are physically meaningful and help where the model knows least. Five-day post-storm weather and location were tested and left out.
- Passed
v5 base: twice the rule's ranking in NSW
With storm context, landscape and fire history, the top 10% of struck cells caught 63% of NSW's 1,426 lightning fires (rule 46%) and 75% of Tasmania's 153 (rule 54%), counting each fire once. In NSW, the 95% ranges from resampled storm days don't overlap.
- Passed
3,000 NSW lightning fires added from the NSW fire history
NSW's open fire history codes cause from 2012–13 onward. Adding it took labelled Australian lightning fires from ~300 to ~3,300, and turned NSW from untested into the strongest result.
- Mixed
Blending North American data: marginal at best
Using the North American model as an input, fine-tuning from it, or training jointly with a continent flag changed Tasmania's ROC-AUC by under 0.02 and didn't lift the top-10% catch. Local records mattered far more.
- Passed
North America: 62% of lightning fires in the top 10% of cells
Trained on US and Canadian seasons 2010–2017, tested on 2018–2020: the top 10% of struck cells held 62% of 50,293 agency-recorded lightning fires, vs 22% for the dry-lightning rule. Canada 2021–2025: 43% vs 21%.
- Didn't work
North American training does not transfer to Australia
A model trained only on North America found 15% of Tasmanian lightning fires in its top 10% (the rule finds 29%), and adding North American data to Australian training made it worse. The method transfers; the model needs local fire records.
- Mixed
Global weather costs ~10 points; the lead over the rule holds
Re-running Tasmania on NASA POWER weather (one source for every country) instead of Australian gauge grids: top-10% catch fell from 62% to 52%, but the dry-lightning rule fell further (44% → 29%).
- Didn't work
Satellite labels can't replace agency records
A model trained on 'new fires seen by satellite after lightning' did no better than chance on agency-confirmed lightning fires. It learned where people burn off. Agency cause records are essential.
- Didn't work
Victoria 2025: no better than lightning density
Trained only on Tasmanian labels, the model did not transfer to Victoria's 2025 season (top 20%: 37% vs 44% for lightning alone). Mainland labels are the fix.
- Passed
v0: Tasmania back-test, 2016–2025 seasons
Each season scored by a model trained only on earlier seasons. Checking the top 10% of struck cells finds 62% of agency-confirmed lightning fires, vs 44% for the dry-lightning rule and 23% for lightning alone.
- Passed
Free lightning data sees most lightning fires
77% of agency-confirmed lightning fires had WWLLN lightning in the same 50 km cell within the 5 days before (87% including neighbouring cells).