CsvEntityIOException because agency_url is missing

[atrawog]

Expected behavior

OTP should treat non essential field in agency.txt as optional.

Observed behavior

09:08:36.080 INFO [main]  (GtfsModule.java:275) reading GTFS bundle at ./GTFS_Steig_2023_obb.zip
09:08:36.094 INFO [main]  (GtfsModule.java:292) Reading entity: org.onebusaway.gtfs.model.Agency
09:08:36.141 ERROR [main]  (OTPMain.java:60) An uncaught error occurred inside OTP: io error: entityType=org.onebusaway.gtfs.model.Agency path=agency.txt lineNumber=4
org.onebusaway.csv_entities.exceptions.CsvEntityIOException: io error: entityType=org.onebusaway.gtfs.model.Agency path=agency.txt lineNumber=4
        at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:161)
        at org.onebusaway.gtfs.serialization.GtfsReader.readEntities(GtfsReader.java:190)
        at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:120)
        at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:115)
        at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:108)
        at org.opentripplanner.gtfs.graphbuilder.GtfsModule.loadBundle(GtfsModule.java:293)
        at org.opentripplanner.gtfs.graphbuilder.GtfsModule.buildGraph(GtfsModule.java:125)
        at org.opentripplanner.graph_builder.GraphBuilder.run(GraphBuilder.java:175)
        at org.opentripplanner.standalone.OTPMain.startOTPServer(OTPMain.java:141)
        at org.opentripplanner.standalone.OTPMain.main(OTPMain.java:55)
Caused by: org.onebusaway.csv_entities.exceptions.MissingRequiredFieldException: missing required field: agency_url
        at org.onebusaway.csv_entities.schema.AbstractFieldMapping.isMissingAndOptional(AbstractFieldMapping.java:100)
        at org.onebusaway.csv_entities.schema.DefaultFieldMapping.translateFromCSVToObject(DefaultFieldMapping.java:43)
        at org.onebusaway.csv_entities.IndividualCsvEntityReader.readEntity(IndividualCsvEntityReader.java:131)
        at org.onebusaway.csv_entities.IndividualCsvEntityReader.handleLine(IndividualCsvEntityReader.java:98)
        at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:157)
        ... 9 common frames omitted
09:08:36.143 INFO [server-shutdown-info]  (OtpStartupInfo.java:44) OTP SHUTTING DOWN (version: 2.5.0-SNAPSHOT, ser.ver.id: 120, commit: 5085fb361c1e91f65407c0cc74811ee34c1d9576, branch: dev-2.x)

Version of OTP used (exact commit hash or JAR name)

Version: 2.5.0-SNAPSHOT, ser.ver.id: 120, commit: 5085fb361c1e91f65407c0cc74811ee34c1d9576, branch: dev-2.x

Data sets in use (links to GTFS and OSM PBF files)

https://mobilitydata.gv.at/en/daten/soll-fahrplandaten-gtfs

Command line used to start OTP

java -Xmx50G -jar ./otp-2.5.0-error-message.jar --build --save . | tee build.log

Reason

This is another episode of Austria likes to crash OTP and is caused by the agency_url and agency_phone missing for some providers in agency.txt.

agency_id,agency_name,agency_url,agency_timezone,agency_lang,agency_phone
"10","Montafonerbahn AG","http://www.montafonerbahn.at/","Europe/Vienna","DE","+43 5556 9000"
"01","OEBB Personenverkehr AG Kundenservice","www.oebb.at","Europe/Vienna","DE","+43 (0)5 1717"
"04","Deutsche Bahn AG","","Europe/Vienna","DE",""
"03","Raaberbahn AG GYSEV Zrt.","","Europe/Vienna","DE",""
"83","Trenitalia","","Europe/Vienna","DE",""
"54","Ceske Drahy","","Europe/Vienna","DE",""
"56","Zeleznice Slovenskej republiky","","Europe/Vienna","DE",""
"02","WESTbahn Management GmbH","","Europe/Vienna","DE",""
"85","Schweizerische Bundesbahnen","","Europe/Vienna","DE",""
"55","Magyar Allamvasutak","","Europe/Vienna","DE",""
"79","Slovenske zeleznice","","Europe/Vienna","DE",""

[leonardehrenfried]

According to the spec agency_url is mandatory.

[leonardehrenfried]

One word of sympathy: As a German I know how hard it is to get the data producers to make their feed compliant.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days

[markstos]

Currently in the US a major US rail provider, Amtrak, is publishing a GTFS feed with a missing agency_url. The agency actually has a URL, Amtrak just published a non-compliant GTFS feed anyway.

Although it's frustrating, I do see that it GTFS producers need to hold up their end of the deal and produce valid GTFS feeds.

[leonardehrenfried]

I really don't want to encourage data producers to be even sloppier when publishing their feeds but out of all the fields in the spec, I do wonder how this one became mandatory.