GTFS realtime stoptime update propagation.

[koch-t]

OTP doesn't seem to propagate delay's as used in the GTFS-RT example at https://developers.google.com/transit/gtfs-realtime/trip-updates

For the same trip instance, three StopTimeUpdates are provided:

delay of 300 seconds for stop_sequence 3
delay of 60 seconds for stop_sequence 8
delay of unspecified duration for stop_sequence 10
This will be interpreted as:

stop_sequences 3,4,5,6,7 have delay of 300 seconds.
stop_sequences 8,9 have delay of 60 seconds.
stop_sequences 10,..,20 have unknown delay.

If such a tripupdate is now given to OTP it will warn about not being able to match the update.

20:43:20.908 WARN (TripUpdate.java:238) Update block matching failed completely.
20:43:20.908 WARN (TripUpdate.java:239) Have a look at the pattern and block:
Stop 00 41839 A0 D73740 >>> Update trip CXX_1483064 Stop #0:ARR_41839 (PREDICTION) A66584 D66584
Stop 01 32900 A73800 D73800 >>> Update trip CXX_1483064 Stop #1:ARR_32900 (PREDICTION) A66700 D66700
Stop 02 47326 A73860 D73860 >>> Update trip CXX_1483064 Stop #2:ARR_47326 (PREDICTION) A66784 D66784
Stop 03 46497 A73920 D73920 >>> Update trip CXX_1483064 Stop #3:ARR_46497 (PREDICTION) A66838 D66838
Stop 04 49107 A73920 D73920 >>> Update trip CXX_1483064 Stop #5:ARR_49493 (PREDICTION) A66937 D66937
Stop 05 49493 A73980 D73980 >>> Update trip CXX_1483064 Stop #8:ARR_33251 (PREDICTION) A67231 D67231
Stop 06 39775 A74280 D74280 >>> Update trip CXX_1483064 Stop #9:ARR_44965 (PREDICTION) A67263 D67263
Stop 07 37054 A74340 D74340 >>> Update trip CXX_1483064 Stop #10:ARR_42463 (PREDICTION) A67301 D67301
Stop 08 33251 A74400 D74400 >>> Update trip CXX_1483064 Stop #12:ARR_33466 (PREDICTION) A67378 D67378
Stop 09 44965 A74400 D74400 >>> Update trip CXX_1483064 Stop #13:ARR_38859 (PREDICTION) A67439 D67439
Stop 10 42463 A74460 D74460 >>> Update trip CXX_1483064 Stop #14:ARR_46516 (PREDICTION) A67560 D67560
Stop 11 35908 A74520 D74520 >>> --
Stop 12 33466 A74580 D74580 >>> --
Stop 13 38859 A74640 D74640 >>> --
Stop 14 46516 A74760 D74760 >>> --
Stop 15 34635 A74880 D74880 >>> --
Stop 16 39916 A75000 D75000 >>> --
Stop 17 49084 A75060 D75060 >>> --
Stop 18 35370 A75120 D75120 >>> --
Stop 19 42957 A75180 D75180 >>> --
Stop 20 44633 A75240 D75240 >>> --
Stop 21 35723 A75300 D75300 >>> --
Stop 22 45416 A75540 D0 >>> --

[abyrd]

I see. Yes, we assume that the updates arrive in contiguous blocks.

[barbeau]

+1 on handling propagation as specified in GTFS-rt spec - AFAICT this issue still exists in the current master branch in TimeTable.update().

Related to this - we were looking at propagating frequency-based real-time Trip Updates (#1347), and had a question on how to handle updates that can cause negative hop times.

For example, for a given trip consider:

stop_A--------stop_B--------stop_C (t1=7:00am)---(t2=?)-----(t3=7:04am)

...where tx = GTFS-rt arrival time, and (?) means no data.

Assume travel time in between stops is 5 minutes (specified in stop_times.txt). When propagating t1 to t2 to create an arrival time at stop_B, you would get t2 = 7:05am. This is a problem, since t2 > t3.

The current code that is doing validation on TripTimes appears to throw out the entire GTFS-rt TripUpdate for all stop_times if one of them creates a negative hop value.

Questions, in light of GTFS-rt spec propagation:

1. Do we continue to validate the real-time data, and if so, what should we throw out? Just the values where the negative time is calculated (e.g., t3, which is presumably erroneous)? Continue to throw out the entire Trip Update? Or...
2. Do we accept real-time data as-is, and just skip over negative hop values at routing time (e.g., by assigning some arbitrary small value to the hop time)?
3. For frequency-based updates and negative hop times, do we treat them the same as schedule-based?

(cc'ing @mona77)

[JordenVerwer]

No, the current code implements delay propagation correctly, although the merge with master happened only recently. The mmri-rt branch was fixed last year.

As for your questions:

1. I'd prefer to throw out the entire trip update, because what else is there to do? Make something up ourselves? That's a bad idea...
2. No, negative hop times are unacceptable in routing, which is why we perform this strict input validation.
3. I see no reason to treat them differently.

[abyrd]

Hi @barbeau,

Our model for the trip update process is that we only apply fully-specified updates, and partial update messages must go through a delay propagation/inference process before they are applied. My feeling is that we should apply a very simple set of rules in that propagation/inference step and throw out any messages that can't be unambiguously turned into reasonable fully-specified updates, complaining loudly in a way that will be useful to upstream data providers and treating them as "no data".

Of course We've got another point of view to take into account here -- providing users with the best possible guess based on poor or partial input data. This is the point of view Google Transit must adopt for example. I don't think we should aim for that currently, we've got a lot of other things to work on and this fuzzy inference stuff could be a rabbit hole.

Getting fully functional RT support in place that can at least handle well-formed coherent updates should be the priority.

[barbeau]

@JordenVerwer @abyrd Ok, thanks, that gives us a baseline for frequency-based real-time updates - we'll throw out any updates that don't apply cleanly in the initial implementation.

@JordenVerwer Could you point me to the code in master that performs the propagation (i.e., applies an update to all stops with stop_sequence > x on a trip, for an update to x)? Maybe I'm not looking in the right place.

@abyrd I can certainly appreciate the complexity that certain types of inference add - you're right, you could spend a ton of time on just that component. And I agree, getting fully functional RT support in place for well-formed updates should be the current priority.

After the 1.0 release when things settle down, though, it may be worth looking at some simple features that a "non-strict" GTFS-rt mode could provide (with the default being a "strict" mode). In my experience, data (real-time especially) is never perfect, and IMO it would be good to have some configurable flexibility in OTP.

[JordenVerwer]

You can find the delay propagation code in lines 455 - 461 of the Timetable class.

[barbeau]

@JordenVerwer Thanks! I missed that the first time. So this issue should be closed, then?

[JordenVerwer]

I suppose so, yeah. @skywave, do you agree that the current implementation is correct?

[miraculixx]

What is the current status on this? Thanks.

[JordenVerwer]

As far as I know, we're still propagating as per the GTFS-RT specification. Did you have any specific reason for asking?

[abyrd]

We should check whether currently any propagation is applied and how.

[barbeau]

Related - @rasbrown opened PR https://github.com/opentripplanner/OpenTripPlanner/pull/3176, which implements propagation across trips within a block (optional via configuration) and is deployed in Tampa, FL as part of the OneBusAway instance there.

@rasbrown In your experience, other than https://github.com/opentripplanner/OpenTripPlanner/pull/3176, does OTP seem to propagation GTFS-realtime delays per the spec within a trip?

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days