sortedTrips doesn't see stoptimes for a trip?

hkvlhm commented 7 years ago

Ran into trouble generating html for Humboldt county.

Simones-MacBook-Air:redwood_test hollykvalheim$ npm list -g gtfs-to-html
/usr/local/lib
└── gtfs-to-html@0.10.1

$ cat config.json

{
  "mongoUrl": "mongodb://localhost:27017/gtfs",
  "agencies": [
    {
      "agency_key": "Sonoma County",
      "url": "http://data.trilliumtransit.com/gtfs/humboldtcounty-ca-us/humboldtcounty-ca-us.zip"
    }
  ],
  "beautify": false,
  "coordinatePrecision": 5,
  "effectiveDate": "July 8, 2016",
  "menuType": "jump",
  "noServiceSymbol": "—",
  "requestStopSymbol": "***",
  "showMap": false,
  "showOnlyTimepoint": true,
  "showStopCity": false,
  "mapboxAccessToken": "YOUR MAPBOX ACCESS TOKEN'",
  "zipOutput": false
}

$ gtfs-to-html |tee gtfs-to-html.log  2>&1 
Starting GTFS import for 1 file
Sonoma County: Downloading GTFS from http://data.trilliumtransit.com/gtfs/humboldtcounty-ca-us/humboldtcounty-ca-us.zip
Sonoma County: Download successful
Sonoma County: Importing GTFS from /Users/hollykvalheim/Timetables/redwood_test/downloads/Sonoma County-gtfs.zip
Sonoma County: Importing - agency.txt - 3 lines imported
Sonoma County: Importing - calendar_dates.txt - 53 lines imported
Sonoma County: Importing - calendar.txt - 8 lines imported
Sonoma County: Importing - fare_attributes.txt - 8 lines imported
Sonoma County: Importing - fare_rules.txt - 37 lines imported
Sonoma County: Importing - feed_info.txt - 1 lines imported
Sonoma County: Importing - frequencies.txt
Sonoma County: Importing - routes.txt - 13 lines imported
Sonoma County: Importing - shapes.txt - 38380 lines imported
Sonoma County: Importing - stop_times.txt - 9531 lines imported
Sonoma County: Importing - stops.txt - 349 lines imported
Sonoma County: Importing - transfers.txt - 8 lines imported
Sonoma County: Importing - trips.txt - 329 lines imported
Sonoma County: Importing - stop_attributes.txt - 810 lines imported
Sonoma County: Post Processing data
Sonoma County: Completed
Completed GTFS import for 1 file
sortedTrips TRILLIUM DEBUG trip:  { _id: 597271abf64eee3b699e3630,
  route_id: '8',
  service_id: 'c_10_b_525_d_31',
  trip_id: 't_44_b_525_tn_0',
  trip_short_name: '',
  trip_headsign: 'Arcata (Westbound)',
  direction_id: 0,
  block_id: '8276',
  shape_id: 'p_470',
  bikes_allowed: null,
  wheelchair_accessible: null,
  trip_type: '',
  agency_key: 'Sonoma County',
  calendar: 
   { _id: 597271a8f64eee3b699d78e0,
     service_id: 'c_10_b_525_d_31',
     service_name: 'Year round calendar (Monday-Friday)',
     monday: 1,
     tuesday: 1,
     wednesday: 1,
     thursday: 1,
     friday: 1,
     saturday: 0,
     sunday: 0,
     start_date: 20140101,
     end_date: 20180101,
     agency_key: 'Sonoma County' },
  dayList: 'Mon-Fri',
  stoptimes: [] }
sortedTrips TRILLIUM DEBUG selectedStoptime:  undefined
TypeError: Cannot read property 'departure_time' of undefined
    at _.sortBy.trip (/usr/local/lib/node_modules/gtfs-to-html/lib/utils.js:63:35)
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3738:18
    at arrayMap (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:660:23)
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3737:24
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3554:27
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:4920:15
    at baseMap (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3553:7)
    at baseOrderBy (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3736:20)
    at Function.<anonymous> (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:9956:14)
    at apply (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:496:27)
    at Function.sortBy (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:6600:16)
    at sortTrips (/usr/local/lib/node_modules/gtfs-to-html/lib/utils.js:53:25)
    at Promise.all.timetablePage.timetables.map (/usr/local/lib/node_modules/gtfs-to-html/lib/utils.js:483:30)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:169:7)

The trip appears to have stoptimes according to mongodb. The same trip exists in multiple agencies, could that be causing trouble?


Simones-MacBook-Air:redwood_test hollykvalheim$ mongo
MongoDB shell version v3.4.4
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.4
Server has startup warnings: 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> use gtfs
switched to db gtfs
> db.stoptimes.find( { trip_id: 't_44_b_525_tn_0' })
{ "_id" : ObjectId("5931da2dd8d70c3040c3b081"), "trip_id" : "t_44_b_525_tn_0", "arrival_time" : "17:30:00", "departure_time" : "17:30:00", "stop_id" : "1260", "stop_sequence" : 2, "stop_headsign" : "", "pickup_type" : 1, "drop_off_type" : 0, "shape_dist_traveled" : 40.8227099812685, "timepoint" : 0, "agency_key" : "Humboldt Transit Authority" }
{ "_id" : ObjectId("5931da2dd8d70c3040c3b082"), "trip_id" : "t_44_b_525_tn_0", "arrival_time" : "16:45:00", "departure_time" : "16:45:00", "stop_id" : "1283", "stop_sequence" : 1, "stop_headsign" : "", "pickup_type" : 0, "drop_off_type" : 1, "shape_dist_traveled" : 0, "timepoint" : 0, "agency_key" : "Humboldt Transit Authority" }
{ "_id" : ObjectId("597271abf64eee3b699e29c2"), "trip_id" : "t_44_b_525_tn_0", "arrival_time" : "17:30:00", "departure_time" : "17:30:00", "stop_id" : "1260", "stop_sequence" : 2, "stop_headsign" : "", "pickup_type" : 1, "drop_off_type" : 0, "shape_dist_traveled" : 40.8227099812685, "timepoint" : 0, "agency_key" : "Sonoma County" }
{ "_id" : ObjectId("597271abf64eee3b699e29c3"), "trip_id" : "t_44_b_525_tn_0", "arrival_time" : "16:45:00", "departure_time" : "16:45:00", "stop_id" : "1283", "stop_sequence" : 1, "stop_headsign" : "", "pickup_type" : 0, "drop_off_type" : 1, "shape_dist_traveled" : 0, "timepoint" : 0, "agency_key" : "Sonoma County" }

hkvlhm commented 7 years ago

Update: we dropped the gtfs database and ran again, so that there is only one agency listed for that trip id. The same error was encountered. Note trillium debug lines were added to source of utils.js

  const sortedTrips = _.sortBy(trips, trip => {
    let selectedStoptime;
    console.log('sortedTrips TRILLIUM DEBUG trip: ', trip); // ED DEBUG
    if (commonStoptime) {
      console.log('sortedTrips TRILLIUM DEBUG commonStoptime: ', commonStoptime); // ED DEBUG
      selectedStoptime = _.find(trip.stoptimes, {stop_id: commonStoptime.stop_id});
    } else {
      selectedStoptime = trip.stoptimes[0];
      console.log('sortedTrips TRILLIUM DEBUG selectedStoptime: ', selectedStoptime); // ED DEBUG
    }
    return moment(selectedStoptime.departure_time, 'HH:mm:ss').unix();
  });

Simones-MacBook-Air:redwood_test hollykvalheim$ gtfs-to-html | tee gtfs-to-html.after-drop-mongo-gtfs-database.log 2>&1
Starting GTFS import for 1 file
Sonoma County: Downloading GTFS from http://data.trilliumtransit.com/gtfs/humboldtcounty-ca-us/humboldtcounty-ca-us.zip
Sonoma County: Download successful
Sonoma County: Importing GTFS from /Users/hollykvalheim/Timetables/redwood_test/downloads/Sonoma County-gtfs.zip
Sonoma County: Importing - agency.txt - 3 lines imported
Sonoma County: Importing - calendar_dates.txt - 53 lines imported
Sonoma County: Importing - calendar.txt - 8 lines imported
Sonoma County: Importing - fare_attributes.txt - 8 lines imported
Sonoma County: Importing - fare_rules.txt - 37 lines imported
Sonoma County: Importing - feed_info.txt - 1 lines imported
Sonoma County: Importing - frequencies.txt
Sonoma County: Importing - routes.txt - 13 lines imported
Sonoma County: Importing - shapes.txt - 38380 lines imported
Sonoma County: Importing - stop_times.txt - 9531 lines imported
Sonoma County: Importing - stops.txt - 349 lines imported
Sonoma County: Importing - transfers.txt - 8 lines imported
Sonoma County: Importing - trips.txt - 329 lines imported
Sonoma County: Importing - stop_attributes.txt - 810 lines imported
Sonoma County: Post Processing data
Sonoma County: Completed
Completed GTFS import for 1 file
sortedTrips TRILLIUM DEBUG trip:  { _id: 59727426a995183ba919aa5f,
  route_id: '8',
  service_id: 'c_10_b_525_d_31',
  trip_id: 't_44_b_525_tn_0',
  trip_short_name: '',
  trip_headsign: 'Arcata (Westbound)',
  direction_id: 0,
  block_id: '8276',
  shape_id: 'p_470',
  bikes_allowed: null,
  wheelchair_accessible: null,
  trip_type: '',
  agency_key: 'Sonoma County',
  calendar: 
   { _id: 59727423a995183ba918ed0f,
     service_id: 'c_10_b_525_d_31',
     service_name: 'Year round calendar (Monday-Friday)',
     monday: 1,
     tuesday: 1,
     wednesday: 1,
     thursday: 1,
     friday: 1,
     saturday: 0,
     sunday: 0,
     start_date: 20140101,
     end_date: 20180101,
     agency_key: 'Sonoma County' },
  dayList: 'Mon-Fri',
  stoptimes: [] }
sortedTrips TRILLIUM DEBUG selectedStoptime:  undefined
TypeError: Cannot read property 'departure_time' of undefined
    at _.sortBy.trip (/usr/local/lib/node_modules/gtfs-to-html/lib/utils.js:63:35)
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3738:18
    at arrayMap (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:660:23)
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3737:24
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3554:27
    at /usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:4920:15
    at baseMap (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3553:7)
    at baseOrderBy (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:3736:20)
    at Function.<anonymous> (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:9956:14)
    at apply (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:496:27)
    at Function.sortBy (/usr/local/lib/node_modules/gtfs-to-html/node_modules/lodash/lodash.js:6600:16)
    at sortTrips (/usr/local/lib/node_modules/gtfs-to-html/lib/utils.js:53:25)
    at Promise.all.timetablePage.timetables.map (/usr/local/lib/node_modules/gtfs-to-html/lib/utils.js:483:30)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:169:7)
Simones-MacBook-Air:redwood_test hollykvalheim$ mongo
MongoDB shell version v3.4.4
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.4
Server has startup warnings: 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] 
2017-07-19T18:12:17.459-0700 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> use gtfs
switched to db gtfs
> db.stoptimes.find( { trip_id: 't_44_b_525_tn_0' })
{ "_id" : ObjectId("59727425a995183ba9199df1"), "trip_id" : "t_44_b_525_tn_0", "arrival_time" : "17:30:00", "departure_time" : "17:30:00", "stop_id" : "1260", "stop_sequence" : 2, "stop_headsign" : "", "pickup_type" : 1, "drop_off_type" : 0, "shape_dist_traveled" : 40.8227099812685, "timepoint" : 0, "agency_key" : "Sonoma County" }
{ "_id" : ObjectId("59727425a995183ba9199df2"), "trip_id" : "t_44_b_525_tn_0", "arrival_time" : "16:45:00", "departure_time" : "16:45:00", "stop_id" : "1283", "stop_sequence" : 1, "stop_headsign" : "", "pickup_type" : 0, "drop_off_type" : 1, "shape_dist_traveled" : 0, "timepoint" : 0, "agency_key" : "Sonoma County" }

brendannee commented 7 years ago

Thanks for posting this.

I pushed updates that solve a few issues:

Now you'll get a better error when you have showOnlyTimepoint= true in the config and there are no timepoints for a trip. This is the case for trip_id t_64_b_525_tn_0 - all stoptimes have timepoint = 0. If you change this in the config, things now work as expected.
I made some updates so that GTFS without timetables.txt and routes where there is no direction_id (direction_id = '') work better. The Humboldt GTFS file only has a file called timetables_new.txt so when running GTFS-to-HTML it just generates timetables for each route and direction_id present. Add a timetables.txt for more control over the routes, directions and days of the week that you want timetables to be created for.

Try it out with 0.10.2 and let me know what you find.

BlinkTagInc / gtfs-to-html

sortedTrips doesn't see stoptimes for a trip? #36