01How do you serve a forecast no two flights share?
Bumpy answers one question: how bumpy will this flight be, minute by minute? The answer depends on the route, the altitude, the aircraft and the hour, so no two flights share one. It also goes stale every hour, when a new forecast arrives.
Three choices make that servable:
- Precompute the weather. One job builds the forecast for the whole world once an hour, so a lookup never waits on weather maths.
- Score on the phone. The server hands out a route plan and a few small pieces of the grid, and the phone turns them into a forecast, so the server does the same small amount of work for every request, however many flights people ask about.
- Say how sure it is. Forecast quality differs by region, so every answer carries its confidence.
This post is the server side of those choices: an hourly job that turns NOAA (the US weather agency) forecasts into a tiled global grid, and a Cloudflare Worker that plans routes and answers lookups. What the phone does with the answer has its own post, Forecasting a flight on the phone. The figures are interactive.
02Architecture
The backend has two parts:
- Ingest job: runs hourly on a dedicated server and builds the forecast grid.
- Cloudflare Worker: backed by D1 and R2, it serves flight lookups, route plans and watches.
The diagram also shows the phone. It isn't backend, but it finishes the job: it downloads the tiles its route needs and scores the route. Select a part to highlight its connections.
03Hourly ingest job
The job starts at 20 minutes past each hour. Three inputs go in, one global grid comes out, and three smaller jobs run off the back of it.
Two rules keep the run dependable:
- It waits for complete inputs. The job only uses a GFS cycle that NOAA has finished posting, and only the last two hours of pilot reports.
- It degrades before it fails. A missing source is logged and the others carry on. The job only fails when no grid can be built at all.
The newest three runs are kept. That number also bounds how far ahead the forecast can be checked against pilot reports. What two weeks of pilot reports look like shows the effect, under Lead time. Logging those reports is a by-product of the job: if it fails, the forecast still ships.
04Forecast sources and coverage
Every source is converted to eddy dissipation rate (EDR), a standard measure of how turbulent the air is. Two forecasts cover the globe:
- Continental US: NOAA's Graphical Turbulence Guidance (GTG). We keep the larger of its clear-air and mountain-wave forecasts, and fade it into the global field at the edge of its area.
- Everywhere else: an estimate from the winds of GFS, NOAA's global weather model, using the Ellrod-Knapp index (vertical shear times deformation) on a 1° grid. It catches jet-stream shear but not storms or mountain waves, and it hasn't been calibrated against observations.
NOAA's global turbulence product (WAFS) is no longer on its public server, which is why the GFS estimate is needed. Answers built from it are flagged so the app can show them with lower confidence.
Thunderstorms are the main gap in both regions. Neither forecast has a storm component. Storms only show up through recent pilot reports, which raise the readings along a route within about 100 km, 4,000 ft and two hours of a report.
Adding storm warnings, radar, lightning and instability data is planned. Learning to forecast turbulence from pilot reports covers how, under The weather models underneath.
In practice this means forecast quality is not uniform: a route over the continental US is sampled from GTG for its whole length, while an oceanic route is mostly sampled from the GFS estimate.
05Regridding and blending
One common grid
The sources arrive on different grids. GTG uses a 3 km grid on a Lambert projection over the US. GFS uses a regular 1° latitude–longitude grid on pressure levels. Before anything is stored, every source is moved onto one common grid: 0.25° spacing and 22 heights, from 1,000 to 45,000 ft.
- Finer than the target: GTG cells are averaged into each 0.25° cell, so a small hot spot is diluted but not lost.
- Coarser than the target: GFS values are interpolated bilinearly.
- Vertical: fields at pressure levels or native heights are interpolated linearly onto the 22 heights.
- One measure per level: where a product gives a combined EDR it wins. Otherwise we keep the larger of the clear-air and mountain-wave forecasts.
Turning winds into EDR
GFS has no turbulence output, so we compute the Ellrod-Knapp index from its winds and map it to EDR with a log-linear fit. The fit is set so the field's spread matches typical cruise climatology: a median near EDR 0.04 and about 1% of points at moderate or worse. That is the same way GTG calibrates its own indices, but our fit has not been checked against pilot reports yet. That check is planned in Learning to forecast turbulence from pilot reports, under Calibrating forecast probabilities.
No seam at the edge
The regional and global fields then have to meet without a seam. A hard switch at the edge of the GTG area would put a step in the forecast that a flight crosses in seconds. Instead GTG's weight falls linearly from 1 to 0 over the last 8 cells (2°) inside its area. Drag the probe across the edge, then switch to a hard cut to see the step it removes.
06Grid and tile format
Each run is a global grid at 0.25° spacing, 22 altitudes and 25 forecast times: hourly for the first 18 hours, then every three hours out to about 36. That's about 600 MB in total.
It's stored as one file per forecast time, split into 10° tiles of 41 × 41 points, one byte per value. The phone can fetch a single tile without the rest of the file. Files never change, because every name includes the run id.
Each value is one byte: EDR in steps of 0.004, so 0 to 254 covers EDR 0 to 1.016 and 255 means no data. That is far finer than the forecast's own accuracy, and it makes a tile small enough to fetch on a weak connection.
Neighbouring tiles share their edge row and column, which is why a tile is 41 × 41 points for 40 × 40 steps. Any point can then be interpolated from a single tile, so the phone never needs a neighbour just to read one value.
How the phone reads a value that falls between grid points is in Forecasting a flight on the phone, under Sampling a tile.
07Flight profile
A forecast for a flight needs to know where the aircraft will be, at what height, and when. Airlines don't publish their flight plans, so we model a typical jet.
- Climb at about 2,000 ft a minute and 280 knots.
- Cruise at a height chosen from the distance: 24,000 ft under 400 km, 32,000 ft under 1,000 km, 36,000 ft under 3,000 km, and 37,000 ft beyond.
- Descend at about 3 nautical miles per 1,000 ft.
- Short hops that can't reach their cruise height top out where the climb and the descent meet.
Times come from the best source the flight lookup has: an actual runway time, then a revised gate time plus typical taxi (15 minutes out, 8 in), then the schedule. When the arrival time is known, cruise speed is stretched or squeezed (between 300 and 560 knots) so the model lands on time. The plan records which source each time came from, so the app can tell a scheduled guess from a real takeoff.
The route is sampled every 50 km. Long-haul routes are capped at 150 points, which spaces them out instead (about 70 km for Singapore to London), so scoring cost stays bounded.
08Scoring, on the server and on the phone
Scoring turns a route plan and the grid into the answer the app shows: a band for each point of the route, the minutes spent in each band, and a headline. The same rules run in two places.
- On the server, in Python, after each hourly run, for watched flights and the 200 most requested, so those lookups come back already scored.
- On the phone, in Swift, for every other flight, so the Worker never has to read weather to answer a request.
Both are tested on the same reference routes and have to agree exactly, so nobody sees two answers for one flight.
The rules themselves are in Forecasting a flight on the phone: how a point is read from a tile, how a pilot report raises a reading, how the bands shift with aircraft size and how the headline is chosen. The server's part in them is the pilot reports. It gathers recent ones with each hourly run and sends those near a route with every lookup, so both scorers work from the same reports.
09Request flow for a flight lookup
A lookup by flight number returns five things: the flight, the route plan, a byte range for each tile the route crosses, nearby pilot reports, and the server's pre-computed score if there is one. The Worker itself never scores.
Flight lookups are cached in D1: for 12 hours when the flight is days away, and 30 minutes near departure. So AeroDataBox is called once per flight per cache window.
What the phone does next, including what happens when one of these calls fails and how long a load takes, is in Forecasting a flight on the phone, under Loading a forecast.
10Caching and serving
Because the weather is precomputed and the phone scores, serving a flight comes down to handing out small files and short answers. Almost all of them can be cached at the network edge, close to the phone.
How long each one is kept depends on how fast it goes stale:
What a phone downloads
A phone downloads only the tiles its route crosses, and each tile is about 37 KB. A tile is needed once for each forecast time the plane spends inside it, so a route usually needs more downloads than the number of 10° boxes it crosses.
Keeping each lookup small
Three more rules keep the work per lookup small:
- Old runs fail cleanly. The three newest runs are kept, so a phone that is mid-download can usually finish. If it asks for a tile from a run that has been removed, the answer says that run has been replaced. The app doesn't start again by itself yet: that load fails and the next one works. It's listed in the iPhone post's Limitations.
- Plans are reused. A flight's route plan is rebuilt only when its times or its aircraft size class change. A new gate or status updates the flight details and nothing else.
- Popular flights are scored ahead of time. Every lookup is counted. After each hourly run the server scores the watched flights and the 200 most requested ones, and stores each score against that run. A lookup for one of those flights returns a finished forecast before the phone has fetched a single tile.
Planning a route reads no weather at all. It needs only the airports, the times and the aircraft, so the Worker's own work per lookup stays the same however large the grid gets.
11Route selection
The forecast is sampled along the route, so the route bounds what the forecast can get right. Transatlantic flights, for example, usually follow the day's organised tracks rather than the great circle. The route finder tries four sources in order and records in the plan which one it used.
Live positions come from adsb.lol, which publishes ADS-B data under the ODbL; the app shows an attribution line wherever it draws a live position or flown track.
12Watched flights and alerts
Every hour, after a new grid is published, the server rescores the flights people are watching plus the most requested flights. The scores are stored, so the next lookup for a popular flight returns instantly.
For watched flights it also decides whether the change is worth a notification. The rule is deliberately quiet:
- First forecast. The first forecast with enough data is always news: "forecast ready".
- Band changes. After that, only a change of headline band counts, and only when the old or new band reaches the threshold the person chose.
- No data, no news. A forecast without enough data never counts as a change.
Each hour's band is recorded even when nothing is sent. So a flight that drifts from light to moderate and back with a moderate threshold produces two notifications, not one per hour. Notification delivery in the app is planned; the server side follows this rule today.
13Failure modes
Most of the system's failure handling comes down to two ideas.
A run is published all at once or not at all. A missing input makes an answer less confident, never wrong without saying so.
| What goes wrong | What the system does | What the user sees |
|---|---|---|
| NOAA hasn't finished posting the newest GFS cycle | Uses the newest cycle that is fully posted, which covers the whole horizon | A forecast up to an hour older, with no gap |
| One source fails or is late | Logs it and builds the grid from the others; a source older than its limit (2 h for GTG, 6 h for GFS) is dropped | The fallback field in that area, flagged as lower confidence where it's the GFS estimate |
| No grid can be built at all | The job fails and the previous run stays live | The previous hour's forecast |
| A phone asks for a forecast mid-publish | Tiles are uploaded first and the pointer to the latest run last; names never change | Always one complete run, never a mix |
| A tile download fails on the phone | Points in that tile have no reading and coverage drops | Fewer bumpy minutes counted, or no headline if under half the flight has data |
| The flight has no arrival time | Models the times from a typical cruise speed and records that it did | A timeline labelled as estimated |
| Thunderstorms | Not in either forecast; only nearby pilot reports raise readings | Storm turbulence can be missed. The plan to add it is in Learning to forecast turbulence from pilot reports |