Empty ID string in GTFS-RT data

[svanhove]

Hi,

I am trying to read the real-time gtfs data in trip_updates.pb in OpenTripPlanner, but OpenTripPlanner is crashing due to the fact that there are no trip_id 's in the data. Here is a dump of trip_updates.pb:

trip {
trip_id: “”
start_date: “20160708”
schedule_relationship: SCHEDULED
route_id: “IC2009″
}
stop_time_update {
stop_sequence: 0
arrival {
delay: 120
time: 1467964440
}
stop_id: “8872009:1″
}
stop_time_update {
stop_sequence: 1
arrival {
delay: 60
time: 1467964860
}
stop_id: “8871100:3″
}

Trip_id seems to be an empty string indeed. Is this intentional? If so, how can I determine what trip the update is about? I know the route_id is given, but OpenTripPlanner uses a different ID for trips and routes. Thank you very much in advance!

[tuukka]

I got this feed read into OpenTripPlanner by hacking its existing fuzzy trip matching code to work based on route_id alone.

[tuukka]

diff --git a/src/main/java/org/opentripplanner/updater/GtfsRealtimeFuzzyTripMatcher.java b/src/main/java/org/opentripplanner/updater/GtfsRealtimeFuzzyTripMatcher.java
index 81153d3..4e0c90b 100644
--- a/src/main/java/org/opentripplanner/updater/GtfsRealtimeFuzzyTripMatcher.java
+++ b/src/main/java/org/opentripplanner/updater/GtfsRealtimeFuzzyTripMatcher.java
@@ -29,19 +29,21 @@ public class GtfsRealtimeFuzzyTripMatcher {
     }

     public TripDescriptor match(String feedId, TripDescriptor trip) {
-        if (trip.hasTripId()) {
+        System.err.println("Trying to match "+trip);
+        if (trip.hasTripId() && !"".equals(trip.getTripId())) {
             // trip_id already exists
             return trip;
         }

-        if (!trip.hasRouteId() || !trip.hasDirectionId() ||
-                !trip.hasStartTime() || !trip.hasStartDate()) {
+        if (!trip.hasRouteId() ||
+                !trip.hasStartDate()) {
             // Could not determine trip_id, returning original TripDescriptor
             return trip;

         AgencyAndId routeId = new AgencyAndId(feedId, trip.getRouteId());
-        int time = StopTimeFieldMappingFactory.getStringAsSeconds(trip.getStartTime());

+        if (trip.hasStartTime()) time = StopTimeFieldMappingFactory.getStringAsSeconds(trip.getStartTime());

             date = ServiceDate.parseString(trip.getStartDate());
@@ -52,7 +54,8 @@

         }
-        int direction = trip.getDirectionId();

+        if (trip.hasDirectionId()) direction = trip.getDirectionId();

         Trip matchedTrip = getTrip(route, direction, time, date);

@@ -62,7 +65,7 @@ public class GtfsRealtimeFuzzyTripMatcher {
             time += 24*60*60;
             matchedTrip = getTrip(route, direction, time, date);
         }
-
+//        System.err.println("... matched trip "+matchedTrip);
         if (matchedTrip == null) {
             return trip;
         }
@@ -76,14 +79,16 @@ public class GtfsRealtimeFuzzyTripMatcher {
                           int startTime, ServiceDate date) {
         BitSet services = index.servicesRunning(date);
         for (TripPattern pattern : index.patternsForRoute.get(route)) {
+//            System.err.println("... matching "+pattern);
             if (pattern.directionId != direction) continue;
             for (TripTimes times : pattern.scheduledTimetable.tripTimes) {
-                if (times.getScheduledDepartureTime(0) == startTime &&
+                if ((startTime == -1 || (times.getScheduledDepartureTime(0) == startTime)) &&
                         services.get(times.serviceCode)) {
                     return times.trip;
-                }
+                } // else System.err.println("no "+date+" "+times.serviceCode);
             }
         }
+        System.err.println("Fuzzy match failed for "+route+" "+direction+" "+startTime+" "+date);
         return null;
     }
 }

[svanhove]

Hi tuukka,

Thanks for your reply. Indeed, these were my thoughts also, that given the data in the realtime gtfs feed, there would be no other way than to work on route_id alone. But surely this is not really the intention according to the specification, espescially since OTP also seems to assume that the trip_id is present. So I was wondering if the lacking trip_id is intentional, or if there are any plans to change this is in the future?

[svanhove]

And thank you very much for the code!

[tuukka]

@pietercolpaert Do you know about plans for this code?

This issue is a duplicate of an issue that @brechtvdv fixed in the spring but I don't know if it was deployed: https://github.com/iRail/brail2gtfs-rt/issues/18

[pietercolpaert]

@tuukka No real plans for this code at this moment: it's out there and will merge pull request and put it into production.

@brechtvdv can you take a look at this issue please?

[pietercolpaert]

@svanhove @tuukka Do you still get this empty trip id string? I actually get a trip id now in the GTFS-RT

[svanhove]

Hi, Indeed, I am getting trip id 's now, but not for all trips. For the current trip_updates.pb (see attached file) I am getting 63 trips with non-empty trip id 's and 6 with an empty string. Any thoughts?

trip_updates.zip

[brechtvdv]

Every time gtfs-rt runs, (a part of) calendar_dates.txt is read. (+/- 4 months) This is because calendar_dates.txt is too big to read every time. When I forget to update this, trip ids are not matched. (A script or database is needed ;) )

The empty strings are because the trip isn't found in the gtfs. Propably a bug with some routes..