Avoid updating static datasets before they are in effect

[cedarbaum]

As a consequence of updating static data daily (#109), Transiter might fetch new schedules before they're in effect. For example, the MTA updated their bus static data around 8/30/2023 but the earliest service start date is 9/3/2023. This could be fixed outside of Transiter by pointing it to a URL that always contains the GTFS data that is in-effect, but it might be nicer to have Transiter inspect the effective service dates and add/delete services as needed.

The logic would be:

• If a service exists, replace it with the newer version iff the effective dates cover the current day
• If a service is new, add it
• If existing services are no longer in effect, delete them

[jamespfennell]

Something like this would be great. I wonder how hard it would be to keep both the old and the new services? Then when querying for scheduled trips, we would use the service that is in the correct date range. This may be highly non-trivial though, and for sure there would be issues with the service IDs of the old and new services colliding.

[cedarbaum]

My intuition here is that some agencies would try to make service and trip IDs unique for each time period (this would need to be more thoroughly researched though). For example, it appears for the MTA bus system that they use new trip IDs for the dataset they published on 8/30. So it's possible we could make some simplifying assumptions (new service IDs, new trip IDs) and make this schedule merging behavior opt-in at the feed-level.

[cedarbaum]

Adding some additional info for handling GTFS schedule transitions for the MTA bus system: https://new.mta.info/developers/gtfs-schedules-transition