staeco / gtfs-stream

Streaming GTFS and GTFS-RT parser for node
MIT License
30 stars 6 forks source link

trip_headsign ignored when containing minus ("-") character #3

Closed BlueskyFR closed 5 years ago

BlueskyFR commented 5 years ago

Some route_id (if not all) are wrong: they are not the same than the ones in routes.txt Also some are missing.

How to reproduce: Use the gtfs.enhanced() example with this file: https://www.data.gouv.fr/fr/datasets/r/e65eaccd-ad10-4fc4-9e10-e886b61ebd9c And try to show the count of objects returned by the "route" data type event, which will be of 31 BUT should actually be of 33. Also, the "route_id" ids will be wrong (different from routes.txt).

Please fix it ASAP :)

BlueskyFR commented 5 years ago

Sorry for opening this, I didn't see that the file had been updated 6 days ago as it is only updated 3 times a year :-/ My bad :-/ Good luck with your project, it is very good!

yocontra commented 5 years ago

@BlueskyFR Alright, feel free to reopen if you still have any issues with it.

As a note - The GTFS enhanced mode is pretty new and still relatively experimental - it might change some more before I set it in stone.

BlueskyFR commented 5 years ago

Hey, I wanted you to test something to be sure this time and not open an issue for nothing :D If a "trip_headsign" has a minus character in it ("-"), it is not created in the result. Example : a trip_headsign equal to "CAMPUS" will return: { route_id: 51, service_id: 1, trip_id: 17227, trip_headsign: 'CAMPUS', shape_id: 98 }

BUT a trip_headsign of "PISCINE-PATINOIRE" will return : { route_id: 58, service_id: 3, trip_id: 104878, direction_id: 1, shape_id: 4610 }

Instead, I observe a direction_id of 1: what is it and why isn't there the normal trip_headsign? This result is obtained using gtfs() but I am facing the same issue using gtfs.enhanced().

Thanks in advance for your help!

yocontra commented 5 years ago

Probably a bug, like I said earlier the enhanced mode is pretty experimental and hasn't been tested on a ton of feeds. Can you upload an example that reproduces the case?

BlueskyFR commented 5 years ago

Sure! You can reproduce the bug by executing this sample:

var gtfs = require('gtfs-stream');
var request = require('request');

request.get("https://www.data.gouv.fr/fr/datasets/r/e65eaccd-ad10-4fc4-9e10-e886b61ebd9c")
.pipe(gtfs/*.enhanced*/()) // The bug occurs with both normal and enhanced modes
.on('data', (obj) => {
  if (obj.type === "trip") {
    let direction = obj.data.trip_headsign;

    // Here, there is a problem: sometimes, the direction variable is undefined.
    // This is maybe when there is a "-" inside.
    // The following should produce output but it is never the case.
    // Extract from the gtfs files :
    /// 58,3,104886,CHAVANOD STADE,0,4607
    /// 58,3,104871,PISCINE-PATINOIRE,1,4610 // This line has a "-" in it BUT it is never displayed.
    // Download yourself the file at line 4 to see in trips.txt
    if (direction && direction.indexOf("-") !== -1) {
      console.log(obj.data.trip_headsign);
    }
  }
});
yocontra commented 5 years ago

Published a fix as 2.0.5 - let me know if you're still having issues with it and I can look into it again. I just looked through it briefly and saw what looked to be the cause, but didn't write a test for it or reproduce it.

BlueskyFR commented 5 years ago

Thanks, works as expected now!