gtfsNeTEx not working for https://opentransportdata.swiss/de/dataset/timetable-2024-gtfs2020/resource/fa70cad5-5714-476d-be90-20c5b8d04fd1

[ue71603]

Imported data and started processing (with NeTEx from the repo as is, no new generation)

C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\venv\Scripts\python.exe C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\GtfsNeTEx.py 
Traceback (most recent call last):
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\GtfsNeTEx.py", line 897, in <module>
    gtfs = GtfsNeTexProfile(conn=duckdb.connect(database='gtfs2.duckdb', read_only=True), serializer=serializer, full=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\GtfsNeTEx.py", line 879, in __init__
    self.incremental()
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\GtfsNeTEx.py", line 853, in incremental
    scheduled_stop_points = self.getScheduledStopPoints(stop_areas, {'query': """select distinct stops.* from trips join stop_times using (trip_id) join stops using (stop_id) where location_type = 0 or location_type is null and route_id = ? order by stop_id;""", 'parameters': (line.private_code.value,)})
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\GtfsNeTEx.py", line 260, in getScheduledStopPoints
    with self.conn.cursor() as cur:
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\GtfsNeTEx.py", line 292, in getScheduledStopPoints
    getRef(stop_areas[stop_area_ref], StopAreaRefStructure)])
           ~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

Process finished with exit code 1

[skinkie]

@ue71603 implement probing with chardet

[ue71603]

will do.

[ue71603]

The problem is different. With the Swiss data set stop_areas remains empty, this results in a problem in line 853 of GtfsNeTEx.py. scheduled_stop_points = self.getScheduledStopPoints(stop_areas, {'query': """select distinct stops.* from trips join stop_times using (trip_id) join stops using (stop_id) where location_type = 0 or location_type is null and route_id = ? order by stop_id;""", 'parameters': (line.private_code.value,)})

[ue71603]

can you have a look. From afar it is unclear to me, where STOP AREAs have any existence in GTFS. I see those as a STOP PLACE hierarchy anyhow.

[skinkie]

I'll take a look.

[ue71603]

you need to do it on the branch, as it needs: https://github.com/skinkie/reference/pull/10/commits/32192362513854f23d02d256f65f47fd5ec549d0

[skinkie]

Please check master.

[ue71603]

works with master