Missing required field in optional schedule file (areas.txt) blocks validator from running

Summary:

I have a static feed that has an invalid areas.txt file (missing wkt field). I want to validate the associated RT data, but the validator throws a fatal Java error about the missing field when downloading the static feed and can't validate the RT data, even though areas isn't actually used by the RT validator as far as I can tell.

Steps to reproduce:

I am experiencing this issue with the MTC 511 regional static feed: http://api.511.org/transit/datafeeds?api_key=<key>&operator_id=RG and associated vehicle positions data: http://api.511.org/transit/vehiclepositions?api_key=<key>&agency=RG

To reproduce,

Start up the web app (I am using Docker, following the README)
Enter the MTC 511 static and vehicle positions feeds listed above
Receive error (details below)

Expected behavior:

I would expect the validator to perhaps throw a non-fatal error about the issue with areas.txt as part of its static feed validation but still be able to validate the RT feed anyway.

Observed behavior:

In the UI, I receive this error:

In the terminal, I get a Java stack trace. The full trace is pasted (collapsed) at the bottom of this post, but the most relevant parts seem to be:

SEVERE null org.onebusaway.csv_entities.exceptions.CsvEntityIOException: io error: entityType=org.onebusaway.gtfs.model.Area path=areas.txt lineNumber=2 ... at edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed.loadGtfsFeedFromDisk(GtfsFeed.java:286) at edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed.postGtfsFeed(GtfsFeed.java:172) ... Caused by: org.onebusaway.csv_entities.exceptions.MissingRequiredFieldException: missing required field: wkt

Platform: OS: MacOS 11.6.1 (Big Sur) Java: Whatever is in the current Docker version as of 2/16/22 (not totally sure?) Browser: Firefox 97.0

Full Java stack trace (click to expand)

``` SEVERE null org.onebusaway.csv_entities.exceptions.CsvEntityIOException: io error: entityType=org.onebusaway.gtfs.model.Area path=areas.txt lineNumber=2 at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:161) at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:120) at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:115) at org.onebusaway.gtfs.serialization.GtfsReader.run(GtfsReader.java:171) at org.onebusaway.gtfs.serialization.GtfsReader.run(GtfsReader.java:159) at edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed.loadGtfsFeedFromDisk(GtfsFeed.java:286) at edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed.postGtfsFeed(GtfsFeed.java:172) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:403) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:334) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:383) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036) at java.lang.Thread.run(Thread.java:748) Caused by: org.onebusaway.csv_entities.exceptions.MissingRequiredFieldException: missing required field: wkt at org.onebusaway.csv_entities.schema.AbstractFieldMapping.isMissingAndOptional(AbstractFieldMapping.java:100) at org.onebusaway.csv_entities.schema.DefaultFieldMapping.translateFromCSVToObject(DefaultFieldMapping.java:43) at org.onebusaway.csv_entities.IndividualCsvEntityReader.readEntity(IndividualCsvEntityReader.java:131) at org.onebusaway.csv_entities.IndividualCsvEntityReader.handleLine(IndividualCsvEntityReader.java:98) at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:157) ... 57 more [qtp1335914322-24] INFO edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed - Downloading GTFS data from http://api.511.org/transit/datafeeds?api_key=94ba868d-fe96-4b9b-b6b4-1df614d96e56&operator_id=RG... [qtp1335914322-24] INFO edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed - GTFS zip file downloaded successfully [qtp1335914322-24] INFO edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed - GTFS URL already exists exists in database - checking if GTFS data has changed... [qtp1335914322-24] INFO edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed - GTFS data hasn't changed since last execution [qtp1335914322-24] INFO edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed - Loading GTFS from downloaded zip file on disk to memory... 2022-02-16T19:43:02.470+0000 SEVERE null org.onebusaway.csv_entities.exceptions.CsvEntityIOException: io error: entityType=org.onebusaway.gtfs.model.Area path=areas.txt lineNumber=2 at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:161) at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:120) at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:115) at org.onebusaway.gtfs.serialization.GtfsReader.run(GtfsReader.java:171) at org.onebusaway.gtfs.serialization.GtfsReader.run(GtfsReader.java:159) at edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed.loadGtfsFeedFromDisk(GtfsFeed.java:286) at edu.usf.cutr.gtfsrtvalidator.api.resource.GtfsFeed.postGtfsFeed(GtfsFeed.java:172) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:403) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:386) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:334) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:221) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036) at java.lang.Thread.run(Thread.java:748) Caused by: org.onebusaway.csv_entities.exceptions.MissingRequiredFieldException: missing required field: wkt at org.onebusaway.csv_entities.schema.AbstractFieldMapping.isMissingAndOptional(AbstractFieldMapping.java:100) at org.onebusaway.csv_entities.schema.DefaultFieldMapping.translateFromCSVToObject(DefaultFieldMapping.java:43) at org.onebusaway.csv_entities.IndividualCsvEntityReader.readEntity(IndividualCsvEntityReader.java:131) at org.onebusaway.csv_entities.IndividualCsvEntityReader.handleLine(IndividualCsvEntityReader.java:98) at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:157) ... 52 more ```

MobilityData / gtfs-realtime-validator

Missing required field in optional schedule file (areas.txt) blocks validator from running #92