search

conveyal / gtfs-lib

A library for loading and saving GTFS feeds of arbitrary size with disk-backed storage
BSD 2-Clause "Simplified" License
71 stars 38 forks source link

Crashes when reading GTFS with bad stop references #96

Open abyrd opened 6 years ago

abyrd commented 6 years ago

This is probably caused by a feed that has stop_times referencing stops that don't exist (I've seen that locally). It makes Analysis unable to load the feed, and it just fails silently from the user point of view.

22:15:47.402 [Thread-12] INFO  com.conveyal.gtfs.GTFSFeed - Generating unique names for patterns
22:15:47.407 [Thread-12] INFO  c.conveyal.taui.persistence.MongoMap - Bundle CH Merge 3 (fixed dates) (5a77849a896fd02b831a4bd0) CH Merge 3 (fixed dates) updated by abyrd+sbb@conveyal.com (sbb)
Exception in thread "Thread-12" com.conveyal.taui.AnalysisServerException
    at com.conveyal.taui.AnalysisServerException.Unknown(AnalysisServerException.java:77)
    at com.conveyal.taui.controllers.BundleController.lambda$null$1(BundleController.java:143)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at com.conveyal.taui.controllers.BundleController.lambda$create$5(BundleController.java:146)
    at java.lang.Thread.run(Thread.java:748)

Also problematic is that the actual exception seems to be swallowed and not reported on the server. If I load the same feed in gtfs-lib locally I see a clearer exception:

317318 [main] INFO com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 633, field 'stop_id': 4256
317318 [main] INFO com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 634, field 'stop_id': 6601
317318 [main] INFO com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 651, field 'stop_id': 6602
317318 [main] INFO com.conveyal.gtfs.GTFSFeed - GTFSError: stop_times line 652, field 'stop_id': 4257
317318 [main] INFO com.conveyal.gtfs.GTFSFeed - GTFSError: transfers line 0: Table is present in zip file, but it has no entries.
317318 [main] INFO com.conveyal.gtfs.GTFSFeed - Building stop to stop times index
317332 [main] INFO com.conveyal.gtfs.GTFSFeed - Building trips per service index
317333 [main] INFO com.conveyal.gtfs.GTFSFeed - Building services per date index
317438 [main] INFO com.conveyal.gtfs.GTFSFeed - Generating unique names for patterns
Exception in thread "main" java.lang.NullPointerException
    at com.conveyal.gtfs.GTFSFeed.lambda$namePatterns$15(GTFSFeed.java:568)
    at com.conveyal.gtfs.GTFSFeed$$Lambda$67/1248334686.accept(Unknown Source)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at com.conveyal.gtfs.GTFSFeed.namePatterns(GTFSFeed.java:567)
    at com.conveyal.gtfs.GTFSFeed.findPatterns(GTFSFeed.java:529)
    at com.conveyal.r5.transit.TransitLayer.loadFromGtfs(TransitLayer.java:208)
    at com.conveyal.r5.transit.TransitLayer.loadFromGtfs(TransitLayer.java:163)
    at com.conveyal.r5.transit.TransportNetwork.fromFiles(TransportNetwork.java:205)
    at com.conveyal.r5.transit.TransportNetwork.fromFiles(TransportNetwork.java:241)
    at com.conveyal.r5.transit.TransportNetwork.fromDirectory(TransportNetwork.java:275)
    at com.conveyal.r5.point_to_point.PointToPointRouterServer.main(PointToPointRouterServer.java:73)
    at com.conveyal.r5.R5Main.main(R5Main.java:29)
abyrd commented 6 years ago

It appears that this code may be entirely removed in current gtfs-lib. We might have to go looking elsewhere for problems like this that mostly just require null checks.