Proposal: Adding Flex rules to the validator

mobilitydataio commented 3 months ago

RULES PROPOSAL

MobilityData has put together a list of validation rules that need to be added or modified now that GTFS-Flex has been adopted into the official specification. (Thanks to @westontrillium for sharing some of Trillium's rules for validating Flex feeds pre-adoption - it was extremely helpful inspiration for this proposal!)

We're seeking feedback on this proposal from the community to see if:

Rules are missing
There should be special considerations for particular rules
Logic should be modified
There are feeds we can use for testing beyond what's already been documented
There are suggestions for how to generically refer to stops.stop_id, locations.id, and location_groups.location_group_id when they all need to be checked in 1 rule

Issues will be made to tackle these specific sections at the beginning of May once feedback is gathered and the proposal is finalized.

SCOPING & TIMELINE

Since there are 24 new rules and 3 rule modifications identified within this document, MobilityData has split up the rules into the following categories, in order of priority:

Pre-existing rules to modify (Phase 1)
routes.txt rules and rules affecting multiple files
stop_times.txt rules
locations.geojson rules
location_groups.txt rules
stops.txt conditionally required rules (blocked)
booking_rules.txt rules

MobilityData commits to completing the pre-existing rule changes before the next validator release, and starting to work through the the rest of the list in the order outlined above. However, given capacity, not all rules will be added in the next release if MobilityData works on this alone.

If you and/or your organization are interested in seeing this work accelerated, you are welcome to contribute! Please let us know here if you would like to support this development effort and we can discuss next steps.

RELATED ISSUES MOBILITYDATA WILL ADDRESS

MESSAGE

GTFS-Flex [Voting ver.] (#433)
* Add 3 "location" files and booking rules file
* Modify stops.stop_id
* Modify routes.continuous_pickup/drop_off
* Modify stop_times.arrival/departure_time
* Modify stop_times.stop_id
* Add stop_times.location_group_id & location_id
* Modify stop_times.stop_sequence
* Add stop_times.start/end_pickup_drop_off_window
* Modify pickup_type & drop_off_type
* Modify stop_times.continuous_pickup/drop_off
* Add pickup/drop_off_booking_rule_id
* Add "On-demand Service Routing Behavior"
* Modify conditions for start/end_pickup_drop_off_window
* Remove unnecessary table name & editorial changes

FILES CHANGED

https://github.com/google/transit/commit/43d8bbc5bcd9feb0a53744727eca03dd0f7d8eb0

https://github.com/google/transit/commit/43d8bbc5bcd9feb0a53744727eca03dd0f7d8eb0

derhuerst commented 3 months ago

Now that Flex v2 has been adopted, will the GTFS Validator support it?

emmambd commented 3 months ago

@derhuerst Yes! The next release will include Flex rules. There will be some discussion here in the next 2 months to solidify the new validation rules and requirements.

emmambd commented 2 months ago

This issue has been updated to include MobilityData's proposal for Flex validation rules. Please read it above and share feedback in the document.

LeoFrachet commented 2 months ago

@emmambd Hi Emma!

The link provided (https://share.mobilitydata.org/flex-validation-rules) doesn't seem to work. Is there an easy way to fix it, or maybe to provide the full URL until it's fixed? Thanks!

emmambd commented 2 months ago

@LeoFrachet Thanks for flagging this! I've updated it with the long form version. Let me know if you still have issues with it.

LeoFrachet commented 2 months ago

Perfect! Thanks, it works!

emmambd commented 1 month ago

Updated this issue with links to the rules MobilityData plans to focus on currently. You can take a look at the specific Github issues to provide more test data or feedback.

LeoFrachet commented 1 month ago

Thanks! While reviewing it I had a question: Is the stops.txt file still required? If they are no stops (only locations), should I output stops.txt empty or can I ignore it?

🤔 In theory it should become optional like every other empty file. That would break many existing GTFS parser, but anyway those parsers will have to be updated to accept stop_times.txt with no stop_id, so I guess it would make sense to make the file optional.

Should I post that kind of feedback here or directly in the document? Lemme know if I should follow another process.

Thanks!

emmambd commented 1 month ago

Hi @LeoFrachet - thanks for raising this! Adding it to the Github thread here was perfect. This consideration came up for us too in thinking through the Flex rules. Our team's created an issue on the spec side to discuss this with the community: https://github.com/google/transit/issues/452.

Once clarified within the spec, we'll make the appropriate change in the validator. You're welcome to share your thoughts and recommendations there.

emmambd commented 2 weeks ago

As an update on the next GTFS Schedule Validator release, this has been delayed due to 1) some unexpected needed performance investigations on Flex rules 2) the MobilityData's team's current focus on releasing feed search for the Mobility Database this summer. As a result, the next validator release will happen later this summer rather than at the end of June.

The focus of the next validator release will be to remove false positive errors for Flex feeds. You can see the list of issues that will be covered in the next release here. Anyone interested in accelerating this work is welcome to contribute to the development effort.

MobilityData / gtfs-validator