transitmatters / t-performance-dash

TransitMatters performance visualizer for the MBTA
https://dashboard.transitmatters.org/
MIT License
51 stars 17 forks source link

Reduce chance of timeout fetching large amounts of data #948

Open devinmatte opened 9 months ago

devinmatte commented 9 months ago

Right now the data dashboard times out when trying to fetch a long range of data

We can potentially improve the performance of the API to reduce the chance of this happening, whether that's by fetching from s3 in parallel, reducing CPU and memory needs, or whatever can help

Examples of data requests which timeout currently https://dashboard.transitmatters.org/red/trips/multi/?from=70069&to=70075&startDate=2022-01-01&endDate=2023-10-31