Machine learning · Nov 2026 · 12 min read

Learning to forecast turbulence from pilot reports

Pairing two thousand pilot reports a day with our forecasts, to train a calibrated model.

  1. 01The question
  2. 02Weather inputs
  3. 03Data
  4. 04The dataset
  5. 05Problem formulation
  6. 06Aircraft size
  7. 07Features
  8. 08Modelling approach
  9. 09Calibration
  10. 10Where it runs
  11. 11Evaluation and rollout
  12. 12Lead time
  13. 13Label quality
  14. 14Failure modes
← Bumpy Blog
Topic
Machine learning, weather
Published
November 2026
Reading time
12 min
Companion post
Serving a turbulence forecast for any flight, to every phone, every hour
Every pilot turbulence report we logged from 1 to 14 November 2026, binned into 2° cells. Dot area is the number of reports; the busiest cells are in blue. Almost all of them are over North America.

01Can pilot reports teach a forecast where it's wrong?

Turbulence forecasts are good where they have been tuned against aircraft measurements, and much rougher elsewhere.

This post covers the data we have so far, how we frame the learning problem, and how we plan to train and test a calibrated model, starting where today's forecast is weakest.

Figure 1. A forecast–observation pairIllustrative
PIREP, 14:05 UTC MOD CHOP · FL350 Forecast at that point light · EDR 0.14
A PIREP gives position, altitude and reported intensity. Paired with the forecast EDR at the same point and time, it becomes one labelled example. Values are illustrative.

02The weather models underneath

Every number in Bumpy starts as a forecast from a weather model. Turbulence has several causes, and each forecast catches some far better than others.

A model can recalibrate what a forecast sees. It can't recover what the forecast never saw.

We use two forecasts:

  • Over North America: GTG. NOAA's Graphical Turbulence Guidance combines several turbulence measures from a detailed weather model, tuned against turbulence measured by airliners. It gives separate clear-air and mountain-wave forecasts, and we keep the larger.
  • Everywhere else: a GFS estimate. We estimate turbulence from the winds of GFS, NOAA's global weather model, using a wind-shear index. It's a fair guide to jet-stream turbulence on a coarse grid, and not much more.

Pick a cause below to see which inputs capture it.

Qualitative

Ratings are our reading of each product, not measured skill.

Storms

Thunderstorms are the biggest gap. Neither forecast we use has a storm component, so storms only reach Bumpy indirectly: through wind shear around large systems, and through reports from pilots who already flew through the bumps.

Four planned inputs would add a direct storm signal: storm warnings (SIGMETs), radar echo tops, lightning and instability (CAPE). They would feed both the model and a separate storm layer in the app, since flights usually route around storm cells rather than through them.

Pilots avoid storms, so reports near them are scarce. Storms will need their own evaluation.

03Data: pairing forecasts with what pilots felt

Pilot reports (PIREPs) give a position, an altitude and how rough the ride was. Pilots also report smooth air.

Each hour, every new report is matched against each forecast run we still hold. We record what that run predicted at the report's exact place and time, plus the strongest forecast nearby. Since several runs are compared with each report, one report yields several examples at different lead times.

Select a run to see the example it produces.

Forecast values are made up
Each example also records the strongest forecast within about 55 km and 2,000 ft. That tolerates small position errors in both the forecast and the report. Smooth-air reports supply the negative examples; without them, any model would lean toward predicting turbulence. Forecast values are illustrative.

04The dataset so far

Logging began on 1 November 2026. These figures cover the first two weeks, up to 14 November.

29,838pilot reports, about 2,200 a day
89,442forecast-vs-report pairs (each report meets up to three runs)
27%of reports say the air was smooth
97.6%of reports are over North America
Figure 2. What pilots reported, 1–14 November 2026
Severe and extreme together are 86 reports, 0.3% of the total, and they often arrive several at a time from one patch of sky.

Where and when reports come from

The reports follow the traffic, and mostly US traffic. The main source is US pilot reports. The rest of the world shows up mainly through automated airline position reports (AIREPs) on oceanic routes.

Figure 3. Report locations
Dot size shows reports per 2° square. Outside North America: Pacific and Asia 332, North Atlantic 231, Europe 52, elsewhere 92.
Figure 4. Reports by altitude
About half of reports are between FL300 and FL400, where airliners cruise. Smooth reports are much more common low down.
Figure 5. Reports by local time of day
Local solar time at the aircraft. Overnight hours are thin, so overnight flights will have the least evidence.

Why the data is sparse

  • It follows the traffic. Reports cluster on busy US routes in daytime. Oceans, nights and most of Asia, Africa and South America barely appear.
  • Bumps get reported, and bad ones are rare. Pilots report bumps more readily than smooth air, yet only 86 reports in two weeks were severe or extreme, often several from one patch of sky.
  • Some pairs are incomplete. 4,164 have no forecast because the report is below the grid's lowest level. The 2,170 AIREPs carry a flight number where the aircraft type would be.
  • Short leads, one global source. Each report is only compared with runs made within about two hours of it. The global WAFS product was unavailable throughout, so every pair outside the GTG area uses the GFS estimate.

How today's forecast scores

For each report we take the newest run made before it, which gives 28,501 fair pairs. The forecast ranks turbulence well but reads low: most moderate-or-worse reports get a forecast below our moderate line.

AUC: how often a bumpy report gets a higher forecast than a smooth one
of moderate-or-worse reports had a forecast at or above 0.22
Each curve is scaled to its own total. Pairs: . Inside the GTG area the median forecast at moderate-or-worse reports is 0.137, well under the 0.22 line. That gap is what calibration is for.

05Problem formulation

Our forecasts use eddy dissipation rate (EDR), a standard measure of how turbulent the air is, independent of the aircraft. Pilot reports use words instead: smooth, light, moderate, severe and the steps between.

We map each word to an approximate EDR for a medium-size airliner, so reports and forecasts share one scale. The first model is then a yes-or-no question: given the forecast and its context, how likely is a pilot to report turbulence at or above a threshold?

Switch the threshold to see which reports count as "yes".

The moderate-or-greater (MOG) label matches the app's moderate band and is the primary target; light-or-greater is secondary. LGT-MOD sits on the MOG boundary and will be evaluated both as positive and as negative.

06From the air to the cabin: aircraft size

EDR describes the air, not the ride. The same patch of turbulence feels very different in a four-seat Cessna and in an A380, because heavier aircraft respond less to the same gusts.

  • Reports depend on the aircraft. A "moderate" from a regional jet and a "moderate" from a widebody aren't the same observation.
  • Today we assume one size. Our forecasts and thresholds assume a medium-size airliner.
  • The plan is five size classes. They follow the way air traffic control already groups aircraft by size and wake.
Figure 6. Aircraft size classes
One representative aircraft per class. The heavy class also includes the 747, 787, A330 and A350. Select a class to use it in the figure below.
Illustrative scaling, not fitted
EDR in the air Felt in this aircraft
One patch of air, five rides. Drag to change the turbulence in the atmosphere, and pick a class to see how its felt intensity shifts. The per-class factors here are made up to show the shape (super is set about 5% gentler than heavy); real ones would be estimated from reports.

We considered a separate model for each aircraft class and decided against it. Pilot reports are already sparse. Split five ways, the smallest and largest classes would have almost no turbulence reports to learn from. Instead, aircraft class is an input to one shared model.

07Features

Each training example starts from the forecast at the report's position and adds the context that changes how far that forecast can be trusted. Some of these are recorded today; others need the forecast pipeline to log more of what it already computes.

GroupFeaturesWhy it matters
ForecastPoint EDR, strongest EDR nearby, forecast sourceThe signal itself, plus tolerance for position error and a flag for which product produced it
WhereFlight level, latitude, terrain heightJet-stream turbulence peaks near cruise levels; mountain waves follow terrain
WhenLead time, monthError grows with lead time; the jet stream strengthens in winter
WhoAircraft size classConverts air turbulence into what the cabin feels
Atmosphere (planned)Wind speed, vertical shear, deformation, Ellrod index, Richardson numberLets a model learn from the ingredients of turbulence, not only from our final number
Convection (planned)Distance to the nearest active storm cell or convective SIGMET, echo top relative to flight level, lightning density, CAPEAdds the storm signal that today's forecast inputs mostly lack

08Modelling approach

Expected gain depends more on the region than on the method. Inside the GTG domain the forecast comes from GTG, which NOAA already calibrates against in-situ EDR, so improvement there is unlikely. Outside it the forecast is the uncalibrated GFS Ellrod-Knapp estimate, which is where we expect calibration to help most.

Relative effort and gain are estimates, not measurements.

    09Calibrating forecast probabilities

    Our first model is a calibration layer. It learns a mapping from forecast EDR to the probability of moderate-or-worse turbulence, using isotonic or logistic regression. There's one mapping per forecast source and altitude band.

    Calibration doesn't move where the forecast puts turbulence. It changes what the numbers mean. It runs when the forecast grid is built, so nothing downstream has to change. Drag the slider to apply it.

    Figure 7. Reliability diagram: forecast probability against observed frequencyIllustrative curves, not our data
    Calibration applied Observed frequency at a 30% forecast
    Points below the diagonal are over-forecast (predicted more often than observed); points above are under-forecast. Curves are illustrative, not from our data.

    10Where the model runs

    The model is applied when the forecast grid is built, once an hour. Its inputs are global weather grids, and its output is written into the same tiles the app already downloads.

    Training is separate and slower: a weekly fit on the paired reports and forecasts collected so far.

    Trainingweekly, minutes per fit (est.) Modeltable or small trees Hourly ingestscores ~1B grid cells a run iPhoneunchanged tiles new pilot reports, paired with forecasts, become next week's training data

    Two rough numbers size the job (back-of-the-envelope estimates, not measurements):

    • About a billion cells an hour. One global run is a 0.25° grid × 22 levels × about 36 hours, and the model scores every cell.
    • Minutes of CPU. A calibration table is one lookup per cell and costs almost nothing. A small tree model over a full run, and each weekly retraining, are estimated at minutes on the same server.

    11Evaluation and rollout

    PIREPs are strongly correlated in space and time: a single weather system can generate dozens of reports. A random row split puts correlated rows in both train and test and overstates skill. We split by time instead, with whole weeks held out.

    Metrics and acceptance criteria:

    • Detection. Probability of detection (POD) and false-alarm ratio (FAR) for moderate-or-greater at the operating threshold.
    • Probabilities. Brier score, ROC AUC and the reliability diagram above.
    • By region. All metrics reported separately for the GTG domain and the GFS region, and by altitude band.
    • Acceptance. Better Brier score than the baseline on held-out weeks in the GFS region, with no regression inside the GTG domain.

    Slicing the results

    A single average hides the cases that matter most. So every metric is broken down by forecast source, altitude band, lead time, aircraft class and season.

    Moderate-or-worse reports are rare, so some slices will be small. We report how many turbulence cases each slice has. Confidence intervals come from a block bootstrap that resamples whole days, since reports from the same day are related.

    Shadow mode before launch

    A model that passes offline tests won't go straight to users. It first runs in shadow mode inside the hourly job. It scores every run alongside the current forecast, and both are logged against the pilot reports that come in afterwards.

    Only when the new model beats the current one on later weeks it has never seen does the forecast switch over. If it later does worse, the previous model is switched back in for the next hourly run.

    12Challenge: forecast lead time

    A report can only be compared with forecast runs we still have when it arrives. We keep the three most recent hourly runs, so in practice every pair falls within about two hours either side of the report.

    Figure 8. Measured lead times, 89,442 pairs
    A run made after the report isn't a fair test. The median fair lead is 29 minutes.

    The app shows forecasts up to about 35 hours ahead, and errors grow with lead time. A model fitted only on short leads would be over-confident further out. Covering longer leads means storing more runs, at roughly 600 MB each.

    Figure 9. Lead-time coverage against disk use
    Runs kept Maximum lead covered Disk used
    Keeping every hourly run costs about 0.6 GB per hour of lead covered. An alternative is to keep every sixth run for 36 hours, which samples leads of roughly 6, 12, 18, 24 and 30 hours for about 3–4 GB extra.

    13Challenge: pilot reports as labels

    97.6%
    of reports are over North America, while the weakest forecast is everywhere else.
    27%
    of reports are smooth air. Pilots report bumps more readily, so negatives are under-represented.
    2,170
    reports (the AIREPs) have no aircraft type, only a flight number. The rest log the type, which becomes the size class.
    86
    severe or extreme reports in two weeks. Calibration needs far fewer of these than a tree model, one reason it comes first.

    Label noise

    Even a perfectly representative set of reports would be noisy:

    • Position. Often approximate, sometimes given relative to a beacon tens of kilometres away.
    • Time. Can lag the encounter by several minutes.
    • Intensity. A judgment call in a busy cockpit, and the line between light and moderate is genuinely fuzzy.

    We handle this three ways:

    1. Tolerance. The nearby-maximum feature absorbs small position and timing errors.
    2. Both readings. Borderline categories are tested both ways rather than forced to one side.
    3. Simple first. We prefer simple models until there's enough data for more.

    Automated in-situ EDR reports from equipped airliners are a natural extension: they are denser, cover smooth air as well as turbulence, and don't depend on how a pilot describes the ride.

    Further out, the phone itself could become a sensor. Its accelerometer measures what a passenger actually feels, in the exact aircraft we're forecasting for. It would have to be strictly opt-in and summarised on the device. We haven't built it, but it's the most direct label for the question Bumpy answers.

    14Failure modes we're watching for

    • Learning the US and applying it everywhere. Most training data comes from US airspace, where the forecast is GTG. A model can look good overall while learning little about the GFS regions it is meant to fix. Per-source metrics and acceptance criteria guard against this.
    • Convective turbulence. Today's forecast inputs carry little storm signal, and the GFS estimate carries none. Calibration can make the numbers more honest, but it cannot add skill the inputs don't contain; that needs the observation-based storm inputs described under The weather models underneath.
    • Seasonal drift. The jet stream moves and strengthens with the seasons. A model fitted in autumn may be miscalibrated by spring, which is one reason to retrain weekly and to evaluate by season.
    • Upstream changes. When NOAA updates a forecast model, the relationship between its output and real turbulence can shift overnight. Shadow-mode logging makes these shifts visible quickly.
    • Over-confidence at long lead times. Until the dataset covers longer leads, the model is evaluated only on short ones, and the app should treat long-range answers with more caution.