The updater config type is unknown: gtfs-http

[mpaine-act]

Expected behavior

router-config.json loads as did in OTP 1.x

Observed behavior

When running the following on Windows 10, the said error occurs.

Version of OTP used (exact commit hash or JAR name)

Version: 2.0.0-rc2-SNAPSHOT Commit: f8f7a9ebe04183e99995010ebfdc33ad68cdcdff

Data sets in use (links to GTFS and OSM PBF files)

1. Using all GTFS from api.511.org (minus a few)
2. Using https://download.geofabrik.de/north-america/us/california/norcal-latest.osm.pbf

Command line used to start OTP

java -Xmx8G -jar O:\target\otp-2.0.0-rc2-SNAPSHOT-shaded.jar --build "$($env:USERPROFILE)\otp\graphs\bayarea" --save

Router config and graph build config JSON

// router-config.json: { "routingDefaults": { "numItineraries": 6, "walkSpeed": 1.2, "stairsReluctance": 4.0, "carDropoffTime": 240 }, "updaters": [ { "type": "stop-time-updater", "frequencySec": 15, "sourceType": "gtfs-http", "url": "https://api.actransit.org/transit/gtfsrt/tripupdates?token=[TOKEN]", "feedId": "9" }, { "type": "real-time-alerts", "frequencySec": 60, "sourceType": "gtfs-http", "url": "https://api.actransit.org/transit/gtfsrt/alerts?token=[TOKEN]", "feedId": "9" }, { "type": "stop-time-updater", "frequencySec": 15, "sourceType": "gtfs-http", "url": "http://api.bart.gov/gtfsrt/tripupdate.aspx", "feedId": "11" }, { "type": "real-time-alerts", "frequencySec": 60, "sourceType": "gtfs-http", "url": "http://api.bart.gov/gtfsrt/alerts.aspx", "feedId": "11" }, { "type": "bike-rental", "frequencySec": 300, "sourceType": "gbfs", "url": "https://gbfs.baywheels.com/gbfs/" }, { "type": "bike-rental", "frequencySec": 300, "sourceType": "gbfs", "url": "https://data.lime.bike/api/partners/v1/gbfs/oakland/gbfs.json" }, { "type": "bike-rental", "frequencySec": 300, "sourceType": "gbfs", "url": "https://s3.amazonaws.com/lyft-lastmile-production-iad/lbs/sfo/" } ] }

//build-config.json: { "areaVisibility": false, "banDiscouragedBiking": false, "banDiscouragedWalking": false, "dataImportReport": false, "distanceBetweenElevationSamples": 10, "elevationUnitMultiplier": 1.0, "embedRouterConfig": false, "extraEdgesStopPlatformLink": false, "fares": null, "fetchElevationUS": false, "islandWithoutStopsMaxSize": 40, "islandWithStopsMaxSize": 5, "matchBusRoutesToStreets": true, "maxDataImportIssuesPerFile": 1000, "maxInterlineDistance": 200, "maxTransferDistance": 2000, "multiThreadElevationCalculations": false, "osmNaming": null, "osmWayPropertySet": "default", "parentStopLinking": true, "platformEntriesLinking": true, "readCachedElevations": true, "staticBikeParkAndRide": false, "staticBikeRental": false, "staticParkAndRide": true, "stationTransfers": false, "storage": null, "streets": true, "subwayAccessTime": 2.0, "transit": true, "useTransfersTxt": true, "writeCachedElevations": false }

Steps to reproduce the problem

1. Build OTP2 JAR files
2. Attempt to build graph using above command line.

[t2gran]

I have looked at this, and tested with the provided configuration and reproduced the error. There is a bit of technical depth here too, so I want to fix both when I first look at this. I hope to have a PR ready for this today.

[t2gran]

I will also fix the case where there is just one source for a given updater, the sourceType should then be optional.

[t2gran]

@mpaine-act I have fixed this issue now on the 2.0-rc branch. You will need to fetch the latest version of the branch to test it. Could you confirm that this now works for you, so we can close the issue?

[mpaine-act]

Building the Index graph completed with the changes. I did receive one error after that:

10:23:19.310 INFO (GraphIndex.java:118) GraphIndex init complete. 10:23:19.311 INFO (Graph.java:630) Index graph complete. 10:23:36.584 INFO (BusRouteStreetMatcher.java:49) Finding corresponding street edges for trip patterns... 10:23:45.267 ERROR (OTPMain.java:45) An uncaught error occurred inside OTP: null java.lang.NullPointerException: null at org.opentripplanner.model.TripPattern.getGeometry(TripPattern.java:189) ~[otp-2.0.0-rc2-SNAPSHOT-shaded.jar:1.1] at org.opentripplanner.graph_builder.module.map.BusRouteStreetMatcher.buildGraph(BusRouteStreetMatcher.java:59) ~[otp-2.0.0-rc2-SNAPSHOT-shaded.jar:1.1] at org.opentripplanner.graph_builder.GraphBuilder.run(GraphBuilder.java:80) ~[otp-2.0.0-rc2-SNAPSHOT-shaded.jar:1.1] at org.opentripplanner.standalone.OTPMain.startOTPServer(OTPMain.java:119) ~[otp-2.0.0-rc2-SNAPSHOT-shaded.jar:1.1] at org.opentripplanner.standalone.OTPMain.main(OTPMain.java:38) ~[otp-2.0.0-rc2-SNAPSHOT-shaded.jar:1.1]

[t2gran]

@mpaine-act I looked at this, and think we can have a quick fix. There is a problem with the data here, and I am not sure what that is, but it seems that OTP is not able to generate "hopGeometries" (the path/lines for the transit vehicle), if you could provide me with a short list of operators (1-5) where you get this problem, then I can reproduce the problem. I will try a few of the first opperators and see if I get the same problem, but I would have to manually download the GTFS files.

[t2gran]

When testing this I run into another problem, the internal model StopLevel class is not Serializable, which causes OTP2 to crash when loading the graph from disk. This class is not used in the data we have tested with, so thank you for providing the config and data files. I will include a fix for this in the next PR.

I have downloaded the GTFS files for all operators by now, and still do not get the same error as you. I do have a fix for the problem - which should work, but it would be nice to reproduce the problem first.

I will provide a PR with the 2 fixes - I think it is safe to merge both, even if I can´t reproduce your problem.

[mpaine-act]

I can provide PowerShell W10 build script which reproduces the issue, if that would help.

[t2gran]

@mpaine-act If you can test the above PR and report back on that that would be great. But since the PR is so simple I hope we can merge it even if you don´t.

I am not sure it is worth the effort to spend a lot of time reproducing the error. Thank you for offering to make a PowerShell script, but I on a Mac and are not sure if it would help me much. I am not running the Real-Time updaters.

I am able to start OTP2 with 23 operators and get back some trips in the bundled UI.

[t2gran]

@mpaine-act The PR is merged so you can test the 2.0-rc branch. If it works feel free to close this issue or let me know.

[mpaine-act]

There is a follow-up when compiling the graph.obj. Using latest from 2.0-rc branch, this error occurs:

15:12:37.365 INFO (BusRouteStreetMatcher.java:49) Finding corresponding street edges for trip patterns... Oct 13, 2020 4:12:02 PM org.geotools.referencing.factory.DeferredAuthorityFactory disposeBackingStore INFO: Disposing class org.geotools.referencing.factory.epsg.hsql.ThreadedHsqlEpsgFactory backing store WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.hsqldb.persist.RAFileNIO (file:/O:/target/otp-2.0.0-rc2-SNAPSHOT-shaded.jar) to method java.nio.DirectByteBuffer.cleaner() WARNING: Please consider reporting this to the maintainers of org.hsqldb.persist.RAFileNIO WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Oct 13, 2020 4:12:02 PM org.hsqldb.persist.Logger logInfoEvent INFO: Database closed

No more logging occurs thereafter.

[t2gran]

@hannesj mentioned that he do not think this is an error, just that the BusRouteStreetMatcher is very slow. So, we should add a progress logging for the BusRouteStreetMatcher.

[t2gran]

@mpaine-act I am closing this issue now since you clearly are past the point of loading updaters - which this issue is about. I did make a new PR to add a progress bar to the bus route street matcher. When I tested it with SF Bay Area GTFS feeds, I noticed another bug, which is also fixed int the PR.

So, thank you for the issue report and testing, and if there are more stuff to fix, just create a new issue.