RRRR doesn't consider shorter time, fails test 2a1

[koch-t]

Test 2a1 has four trips along the same route/journeypattern. In both cases there is a longer and shorter variant, the shorter variant departs later but always arrives at the destination first. Ergo the router should always give an arrivaltime of 00:02:00 or 00:05:00. When departing at 00:00:00 we expect to get a intinrary via 2a1|short|1 to arrive at 00:02:00, if that is not possible we expect a trip via 2a1|long|2 to arrive at 00:06:00

trips.txt
route_id,service_id,trip_id
2a1|short,ignore,2a1|short|1
2a1|short,ignore,2a1|short|2
2a1|long,ignore,2a1|long|1
2a1|long,ignore,2a1|long|2

stop_times.txt
trip_id,arrival_time,departure_time,stop_id,stop_sequence
2a1|long|1,00:01:00,00:01:00,2a1,1
2a1|long|1,00:03:00,00:03:00,2a2,2

2a1|short|1,00:01:00,00:01:00,2a1,1
2a1|short|1,00:02:00,00:02:00,2a2,2

2a1|long|2,00:05:00,00:05:00,2a1,1
2a1|long|2,00:06:00,00:06:00,2a2,2

2a1|short|2,00:05:00,00:05:00,2a1,1
2a1|short|2,00:07:00,00:07:00,2a2,2

NOTE: don't look at the route_id, which is set to the long because it 's the same route and we assign route_id per route.

CONTEXT
n_stops: 2
n_routes: 1

STOPS
stop 0 at lat 0.000000 lon 0.000000
served by routes 0 
stop 1 at lat 0.000000 lon 0.000000
served by routes 0 

ROUTES
route 0
serves stops 0 1 

STOPIDS
stop 000 has id Stop 2a1 
stop 001 has id Stop 2a2 

ROUTEIDS, TRIPIDS
route 000 has id bus long and first trip id LEV1���R 

-- Router Request --
from:  Stop 2a1 [0]
to:    Stop 2a2 [1]
date:  2014-01-01
time:  00:00:00 [1388530800]
speed: 1.500000 m/sec
arrive-by: false
max xfers: 5
max time:     --   

origin_time 00:00:00 
Initializing router state 
stop 0 was marked as updated 
  applying transfer at 0 (Stop 2a1) 
  flagging route 0 at stop 0
  route running

Router states:
Stop name [sindex]  round 0  round 1  round 2  round 3  round 4  round 5
 Stop 2a1 [     0] 00:00:00    --       --       --       --       --   

round 0
  route 0: bus long

Route details for 'bus long' [0] (n_stops 2, n_trips 4)
stop sequence, stop name (index), departures  
   0                            Stop 2a1 [000000] : 00:01:00 -1D 00:01:00 -1D 00:05:00 -1D 00:05:00 -1D 
   1                            Stop 2a2 [000001] : 00:03:00 -1D 00:02:00 -1D 00:06:00 -1D 00:07:00 -1D 

    stop  0 [0] 00:00:00 Stop 2a1
hit previously-reached stop 0

Route details for 'bus long' [0] (n_stops 2, n_trips 4)
stop sequence, stop name (index), departures  
   0                            Stop 2a1 [000000] : 00:01:00 -1D 00:01:00 -1D 00:05:00 -1D 00:05:00 -1D 
   1                            Stop 2a2 [000001] : 00:03:00 -1D 00:02:00 -1D 00:06:00 -1D 00:07:00 -1D 

    board option 0 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 1 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 2 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 3 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 0 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 1 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 2 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 3 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 0 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    board option 1 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    board option 2 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    board option 3 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    boarding trip 0 at 00:01:00 
    stop  1 [1]    --    Stop 2a2
    on board trip 0 considering time 00:03:00 
    setting stop to 00:03:00 
stop 1 was marked as updated 
  applying transfer at 1 (Stop 2a2) 
  flagging route 0 at stop 1
  route running
round 1
  route 0: bus long

Route details for 'bus long' [0] (n_stops 2, n_trips 4)
stop sequence, stop name (index), departures  
   0                            Stop 2a1 [000000] : 00:01:00 -1D 00:01:00 -1D 00:05:00 -1D 00:05:00 -1D 
   1                            Stop 2a2 [000001] : 00:03:00 -1D 00:02:00 -1D 00:06:00 -1D 00:07:00 -1D 

    stop  0 [0] 00:00:00 Stop 2a1
hit previously-reached stop 0

Route details for 'bus long' [0] (n_stops 2, n_trips 4)
stop sequence, stop name (index), departures  
   0                            Stop 2a1 [000000] : 00:01:00 -1D 00:01:00 -1D 00:05:00 -1D 00:05:00 -1D 
   1                            Stop 2a2 [000001] : 00:03:00 -1D 00:02:00 -1D 00:06:00 -1D 00:07:00 -1D 

    board option 0 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 1 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 2 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 3 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000010000

    board option 0 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 1 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 2 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 3 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000000100000

    board option 0 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    board option 1 at 00:01:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    board option 2 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    board option 3 at 00:05:00 -1D 
00000000000000000000000000100000
00000000000000000000000001000000

    boarding trip 0 at 00:01:00 
    stop  1 [1] 00:03:00 Stop 2a2
    on board trip 0 considering time 00:03:00 
    (no improvement)
round 2
round 3
round 4
round 5

A 1 VEHICLES 
bus long;Stop 2a1;00:01:00;Stop 2a2;00:03:00

[abyrd]

This is not surprising. The original algorithm assumes trips on a route are FIFO. This is not a sound assumption when real-time updates are in effect, so we have already improved on this by not assuming that trips are FIFO up to the boarding stop (all trips are considered at the boarding stop). However, downstreamof boarding we continue to assume they are FIFO.

This is equivalent to the assumption that the passenger will board the first bus that arrives on a line rather than waiting for the second bus on a given line based on travel advice telling them the second bus will overtake the first. Honestly 1) I doubt anyone would believe such advice, and 2) do you know of any real-world example of overtaking by vehicles on the same JourneyPattern on the same day?

-Andrew

[koch-t]

You're right.

[koch-t]

Then again this can really happen when we apply real-time data.

[abyrd]

It can happen but it near-certainly means the realtime data is messed up, so we are effectively assuming the passenger will take the first vehicle of that line which passes, which is exactly what every normal passenger will do.

[koch-t]

Well it doesn't happen that often but overtaking within a journeypattern surely can happen without messed up real-time data. Think of a bus that breaks down but continues after 10 minutes on a very high-frequency line.

[abyrd]

Yes it does happen. But how should/would that affect planning? At all locations upstream of the breakdown, the breakdown situation is not known because it has not happened yet. At all locations downstream of the breakdown, the trips are still FIFO. This is the behavior I'm counting on: make sure a trip has not fallen behind (do not assume trip ordering is the same later in the trip as it is at the beginning of the trip), but also assume that once the delayed vehicle has resumed its course it will not pass vehicles ahead of it on the route, or more importantly assume that the passenger's mental model is such, and that there's no point defying it.

[koch-t]

There is also another way of "fixing" this test, at least making sure there is the chance of this happening in real life is almost zero. Just add the route_id to pattern_signature in gtfsdb.py, we need distinct route_id's anyway per journey pattern if we want to continue assigning metadata to a route.

--- ../bliksem2/gtfsdb.py   2013-07-29 18:54:27.928960423 +0200
+++ gtfsdb.py   2013-07-31 10:28:03.997866574 +0200
@@ -397,13 +397,14 @@
             if reporter and i%(n_trips//50+1)==0: reporter.write( "%d/%d trips grouped by %d patterns\n"%(i,n_trips,len(bundles)))

             d = self.get_cursor()
-            d.execute( "SELECT trip_id, arrival_time, departure_time, stop_id FROM stop_times WHERE trip_id=? AND arrival_time NOT NULL AND departure_time NOT NULL ORDER BY stop_sequence", (trip_id,) )
+            d.execute( "SELECT trip_id, arrival_time, departure_time, stop_id,route_id FROM stop_times LEFT JOIN trips using (trip_id) WHERE trip_id = ? AND arrival_time NOT NULL AND departure_time NOT NULL ORDER BY stop_sequence", (trip_id,) )

             stop_times = list(d)

-            stop_ids = [stop_id for trip_id, arrival_time, departure_time, stop_id in stop_times]
-            dwells = [departure_time-arrival_time for trip_id, arrival_time, departure_time, stop_id in stop_times]
-            pattern_signature = (tuple(stop_ids), tuple(dwells))
+            stop_ids = [stop_id for trip_id, arrival_time, departure_time, stop_id,route_id in stop_times]
+            dwells = [departure_time-arrival_time for trip_id, arrival_time, departure_time, stop_id,route_id in stop_times]
+            route_id = [route_id for trip_id, arrival_time, departure_time, stop_id,route_id in stop_times][0]
+            pattern_signature = (tuple(stop_ids), tuple(dwells),route_id)

             if pattern_signature not in patterns:
                 pattern = Pattern( len(patterns), stop_ids, dwells )

[abyrd]

...but "route" has no semantics at all in GTFS. This change would be counting on Routes to have a meaningful definition.

[abyrd]

Well, it might not hurt to add GTFS route to the journeypattern key, as this would justify storing only one route name / headsign etc. per journeypattern (which we already do). It would be pretty insane for a feed producer to create lots of trips with the same sequence of stops and assign them to many different routes.

[koch-t]

Well there is semantic regarding vehicle_type for example. The most real-life example i can think of is in Amsterdam where tram and busstops share the same stop_id's. Theoretically this could happen that a tram and a bus route go along the same stops but the tram is faster.

/reply to second reply.

Yes but that is the point, adding route_id,this will add a negligible amount of extra routes but will counter a lot of freak scenarios such as the one in this test. The RID exporter also make trip bundles on the condition, sequence-of-stops, sequence-of-pickupallowed,sequence-of-dropoffallowed,line_id,tripheadsign.

[abyrd]

OK you convinced me :+1: I think it's highly unlikely to have two different routes hitting all the same stops in the same order, one faster than the other, but if it does exist they're likely to be marked as different modes/routes, and it's just a more watertight design.

[abyrd]

So the test should also be altered to, for example, have the faster overtaking trip be a different mode (Rail) than the slower one (bus), in which case it will pass. And it needs a comment to that effect.