search

a-b-street / 15m

New version of the 15-minute neighborhood tool
https://a-b-street.github.io/15m/
Apache License 2.0
6 stars 0 forks source link

Start grabbing GTFS data from FGB files on the web #7

Closed dabreegster closed 4 months ago

dabreegster commented 4 months ago

Part of #4. The approach is:

1) Offline, assemble one massive GTFS feed covering ideally the world. Right now, it's just the UK, thanks to https://data.bus-data.dft.gov.uk/. 2) Run a tool that extracts relevant info from the GTFS and expresses as a flatgeobuf file 3) When the user imports an area on the web (dynamically from OSM data, thanks to the Overpass API), make the browser also query relevant areas of the FGB file and build the public transit graph too!

The "performance" is comically bad right now. Just the Elephant & Castle area triggers about 770 requests and sucks down 18MB. But not bad for a low-effort simple start!

The main missing thing is clipping; I'll try to add that now. Any trip intersecting the boundary area will have all stops included. So for part of central London, we wind up with stops very very far away: image This also messes up the routing.

Also before merging, I'll upload the FGB file and hook it up to the GH Pages demo... but maybe only if I can get the performance to be better, so I'm not burning through bandwidth ridiculously.

Encoding

What's in the FGB file exactly? Per "route variant", a linestring, with one point per stop. (The full route shape isn't included yet.) What's a "route variant"? In GTFS, "route 5" has many trips, some of them with potentially different sequences of stops (like an express / weekend / full service). To express a route geometrically, we need to decide the stops included, so we use the sequence of stops as the primary key.

For each of these route variants with a linestring, some JSON-encoded data:

dabreegster commented 4 months ago

Perf update: after using the most popular Rust compression crate on the properties, the full FGB file goes from 270MB to 100MB. Loading the same area now takes 935 requests (!!) but only 6.5MB instead of 18.

dabreegster commented 4 months ago

OK, I implemented clipping. Some of the routes near the map boundary are still odd, but I'll merge now. I've uploaded the UK GTFS to https://assets.od2net.org/severance_pbfs/gtfs.fgb (you can download the full thing and open in QGIS if you're curious) and wired up the web demo to use it.

Loads of work to do, but proof-of-concept of PT routing "anywhere" entirely within a browser!