ad-freiburg / pfaedle

Precise map-matching for public transit feeds. Generates high-quality GTFS shapes from OSM data.
GNU General Public License v3.0
203 stars 29 forks source link

Could not parse input GTFS feed "stop_name" #22

Closed 2803media closed 3 years ago

2803media commented 3 years ago

Hi is there a way to bypass those error?

[2020-11-19 10:17:17.249] ERROR: Could not parse input GTFS feed, reason was:
gtfs/fr-sw_gtfs/stops.txt:10943: in field 'stop_name', expected non-empty string

gtfs file here : https://navitia.opendatasoft.com/explore/dataset/fr-sw/files/233a49a3667ccbff85623e400b3ea969/download/

Thanks

hbruch commented 3 years ago

This is rather an error in the GTFS file, as according to the GTFS spec, for stops of location_type=0 (stop), 1 (station) and 2 (entrances/exits) stop_name is required:

https://developers.google.com/transit/gtfs/reference#stopstxt:

stop_name:
• Required for locations which are stops (location_type=0), stations (location_type=1) or entrances/exits (location_type=2).

For row 10943, the stop_name is indeed empty while it should not, same as it's child stop in row 39282:

stop_id,stop_name,stop_desc,stop_lat,stop_lon,zone_id,stop_url,location_type,parent_station,wheelchair_boarding
StopArea:STEStopArea:OCE87699298,"",,0,0,,,1,,
StopPoint:STEStopPoint:OCECarTER-87699298,"","",0,0,,,0,StopArea:STEStopArea:OCE87699298,

To fix your issue, you might preprocess the GTFS and patch empty stop_names.

2803media commented 3 years ago

Thanks I understand this point I will try some tools to clean the GTFS, I found https://github.com/patrickbr/gtfstidy any other recommandations?

tx

hbruch commented 3 years ago

I think gtfstidy won't fix missing required values. But unzipping the zip, fixing the stops.txt, and zipping it again should work for a quick one time fix.

According to GTFSVTOR validator, this feed has 20 stop_name violations. The following grep commands list those 20 records:

$ grep '"",,' stops.txt
StopArea:STEStopArea:OCE87699298,"",,0,0,,,1,,
StopArea:STEStopArea:OCE87279166,"",,46.834999,1.565542,,,1,,
StopArea:STEStopArea:OCE87644112,"",,47.219605,0.881187,,,1,,
StopArea:STEStopArea:OCE87110288,"",,47.34836,0.660652,,,1,,
StopArea:STEStopArea:OCE87689737,"",,47.394142,0.671523,,,1,,
StopArea:SINStopArea:OCE87644112,"",,47.219605,0.881187,,,1,,
StopArea:SINStopArea:OCE87110288,"",,47.34836,0.660652,,,1,,
StopArea:SINStopArea:OCE87699298,"",,0,0,,,1,,
StopArea:SINStopArea:OCE87279166,"",,46.834999,1.565542,,,1,,
StopArea:SINStopArea:OCE87689737,"",,47.394142,0.671523,,,1,,
Holgers-MBP:fr-sw_gtfs hbruch$ grep '"","",' stops.txt
StopPoint:STEStopPoint:OCECarTER-87699298,"","",0,0,,,0,StopArea:STEStopArea:OCE87699298,
StopPoint:STEStopPoint:OCECarTER-87279166,"","",46.834999,1.565542,,,0,StopArea:STEStopArea:OCE87279166,
StopPoint:STEStopPoint:OCECarTER-87644112,"","",47.219605,0.881187,,,0,StopArea:STEStopArea:OCE87644112,
StopPoint:STEStopPoint:OCECarTER-87110288,"","",47.34836,0.660652,,,0,StopArea:STEStopArea:OCE87110288,
StopPoint:STEStopPoint:OCECarTER-87689737,"","",47.394142,0.671523,,,0,StopArea:STEStopArea:OCE87689737,
StopPoint:SINStopPoint:OCECarTER-87644112,"","",47.219605,0.881187,,,0,StopArea:SINStopArea:OCE87644112,
StopPoint:SINStopPoint:OCECarTER-87110288,"","",47.34836,0.660652,,,0,StopArea:SINStopArea:OCE87110288,
StopPoint:SINStopPoint:OCECarTER-87699298,"","",0,0,,,0,StopArea:SINStopArea:OCE87699298,
StopPoint:SINStopPoint:OCECarTER-87279166,"","",46.834999,1.565542,,,0,StopArea:SINStopArea:OCE87279166,
StopPoint:SINStopPoint:OCECarTER-87689737,"","",47.394142,0.671523,,,0,StopArea:SINStopArea:OCE87689737,

To automate GTFS preprocessing, you might consider OneBusAway's transformer tool.

While I don't think, there already exists an update strategy to fix stop names, this one is fairly close and could serve as an example.

2803media commented 3 years ago

Thanks @hbruch for your kind input I'll digg on this!

patrickbr commented 3 years ago

Just a minor follow up on this, it is actually possible to fix this error with gtfstidy. You can use the options -e --empty-str-repl "-". -e will replace erroneous fields with their default value, and --empty-str-repl "-" will use - as a replacement for empty strings which shouldn't be empty. You can of course use any other string you like, e.g. "(No name)".

So if feed.zip is your feed, just use

gtfstidy -e --empty-str-repl "-" -o feed-fixed.zip feed.zip

2803media commented 3 years ago

Very interesting @patrickbr ! to install on mac I only need to install go before, with brew for example?

patrickbr commented 3 years ago

Indeed, after you have installed go, you can do a

go get github.com/patrickbr/gtfstidy