Make Shapes a recommended file in GTFS

isabelle-dr commented 4 months ago

Context

As part of our efforts to merge the GTFS Best Practices into the spec, we are:

migrating the current Best Practices into the spec based on community consensus (plan for Schedule, plan for Realtime).
evaluating all outstanding issues and PRs that existed on the Best Practice repos, and proposing new Best Practices to be added directly into the spec, if still relevant.

Scope for this issue

This issue picks-up on https://github.com/MobilityData/GTFS_Schedule_Best-Practices/issues/37 by @e-lo aiming at making shapes.txt a recommended file in GTFS.

Considerations

What are other GTFS files currently recommended?

feed_info.txt is a recommended file. If it's not included in a GTFS dataset, the canonical GTFS Schedule Validator triggers a missing_recommended_file.

Why aren't we just making all optional files/fields recommended?

Certain Optional files/fields are dependent on the service (e. g. timeframes.txt is for modeling fares based on time of day), whereas other files can and should always be added regardless of the type of service being modeled (e. g. feed_info.txt). We think there is value in calling out the latter explicitly using the term recommend to promote higher quality GTFS.

With the addition of GTFS Flex, if we model a zone-based demand-responsive service, we don't need shapes.txt. We could not use the Recommended presence requirement the same way we do for feed_info.txt. That being said, we could add it in its description (similarly to the field min_width), and write custom logic for triggering missing_recommended_file in the validator only when relevant.

Would you support this change? Tagging folks who engaged in adding the Recommended presence into GTFS @e-lo @antrim @bdferris-v2 @NomeQ @gcamp @evansiroky @markstos @westontrillium @derhuerst @doconnoronca

skinkie commented 4 months ago

I don't think this should be recommended. Shapes are not required for a journey planner to work, it is a presentation thing specifically targeted for journey planners that present their results on a map. Any timetable journey planner before Google Transit or OpenTripPlanner never did that in the first place.

doconnoronca commented 4 months ago

As a transit predictor who calculates its own predictions, TransSee uses the shape data to identify and store exactly how far along a trip each vehicle is. While TransSee can work without it, it works much better if it is included. Fortunately most agencies do include it.

skinkie commented 4 months ago

As a transit predictor who calculates its own predictions, TransSee uses the shape data to identify and store exactly how far along a trip each vehicle is. While TransSee can work without it, it works much better if it is included. Fortunately most agencies do include it.

Then why don't you calculate the shapes when they are not available?

doconnoronca commented 4 months ago

Then why don't you calculate the shapes when they are not available?

I looked into that in the past. Using a road trip planner didn't always work because buses are sometimes allowed to go on roads other vehicles can't. I looked into algorithms that would allow me to make it from combining recorded vehicle locations but I couldn't find one that would make sharp corners.

If this problem was more common I would work on it more.

Transit App seems to do this when detecting detours, but only for Montreal STM and they pushed for Trip-Modifications so they can get the information from the agency instead.

isabelle-dr commented 4 months ago

It might also be worth mentioning that 85% of the GTFS Schedule datasets on the Mobility Database have shapes.txt, which is more than feed_info.txt (which is present 62% of the time). This is another reason why we picked it up from the Best Practice repo: making shapes.txt recommended seems to be reflecting what we're seeing in practice.

gcamp commented 4 months ago

At Transit, we detect detours but also create shapes when they are not present. Even with that in mind, there's significant value for us to have a shape as defined by the agency because if the vehicle takes a non-optimal path between two stops (which happens sometimes), there's no way for us to know that without a shape.

I agree shapes is not used for every use case, but I still think it should be recommended because there's clear use case for it.

markstos commented 4 months ago

Given the 85% adoption rate, I support making shapes recommended. Recommended is still not required.

evansiroky commented 4 months ago

+1 from Caltrans, we use shapes a lot in lots of analyses and believe it provides significant value to customers wanting to view their journey on a map.

westontrillium commented 4 months ago

In support of this change. @evansiroky hit the nail on the head: Riders should have access to the vehicle's scheduled path of travel as defined by the agency.

Obvious caveat for zone-based services. :)

bdferris-v2 commented 4 months ago

+1 to making shapes.txt recommended

We also see a lot of value in shapes, echoing comments from others:

User feedback has repeatedly shown the value of showing accurate shapes on a map-based presentation
Shapes are useful for other data quality validations, including evaluating stop location and orientation
Shapes can be useful in combination with real-time vehicle position feeds, though I acknowledge you can also learn shapes from VP data.

isabelle-dr commented 3 months ago

Heads-up that a vote has been opened in the associated PR.

google / transit