Transformation CH SKilifts-> EPIP -> NeTEX

[ue71603]

I tried to use the first swiss skilifts. They exist as NeTEx and I want to create a GTFS. There are some data errors, on which I am working. However, the process still stops.

netex.zip

Full run:

REM CH Skilift -> EPIP --> GTFS
del C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/*.duckdb
python netex_stats.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/skilift/netex.xml
python swiss_to_db.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/skilift/netex.xml C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/swiss-import.duckdb
python epip_db_to_db.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/swiss-import.duckdb C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex-import-epip.duckdb
python epip_db_to_xml.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex-import.duckdb C:/Users/ue71603/MG_Daten/conversion/swiss_skilift//netex-import-epip.duckdb C:/Users/ue71603/MG_Daten/conversion/swiss_skilift//netex.xml
python netex_stats.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex.xml
del C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/*.duckdb
python netex_to_db.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex.xml  C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex-database.duckdb
python netex_db_to_gtfs.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex-database.duckdb C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/gtfs_output.zip
del C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/*.duckdb

error in python epip_db_to_db.py C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/swiss-import.duckdb C:/Users/ue71603/MG_Daten/conversion/swiss_skilift/netex-import-epip.duckdb

Traceback (most recent call last):
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\transformers\epip.py", line 398, in epip_service_journey_generator
    with sqlite3.connect(write_database) as write_con:
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\transformers\epip.py", line 404, in epip_service_journey_generator
    write_generator(write_con, ServiceJourney, query(read_con), True)
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\netexio\dbaccess.py", line 255, in write_generator
    for a in _prepare3(generator, objectname):
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\netexio\dbaccess.py", line 231, in _prepare3
    for obj in generator3:
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\transformers\epip.py", line 391, in query
    yield process(sj, read_database, write_database, generator_defaults)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\transformers\epip.py", line 375, in process
    service_journey_ac_to_day_type(sj, availability_conditions, day_types, uic_operating_periods, day_type_assignments)
  File "C:\Users\ue71603\MG_Daten\github\reference\gtfs-netex-test\transformers\epip.py", line 296, in service_journey_ac_to_day_type
    valid_days[0]),
    ~~~~~~~~~~^^^
IndexError: list index out of range

[skinkie]

@ue71603 you might want to review the AvailabilityCondition's mentioned in the file. While valid I see duplications of AvailabilityConditions that are never valid. My guess the problem is: if the sequence is never valid, what would the OperatingPeriod look like? I made a deliberate choice not just to trust the chain of availability conditions due to the fact that they can be positive and negative, effectively this materialises all stacked AvailabilityConditions in a single UicOperatingPeriod. What we could do: if it is never valid, take the first day as first and last day, and mark it zero. Raising a warning is likely oppertune.

In addition, we currently don't handle TemplateServiceJourney. I don't know what profile it is or what the intention is. If that should be converted to frequencies.txt that is something we certainly don't support yet (not even loading them).

[ue71603]

TemplateServiceJourney: Yes we are talking frequency based stuff. Wasn't that in the main swiss data already? AvailabilityCondition: I will check

[skinkie]

This file has issues too. For example <PublicCode/>, but it also lacks an OperatorRef.

Now you could say: you can derive this by ResponsibilitySet, I don't like how that route is modeled... I have implemented the ResponsibilitySet method, to find out that ch:1:ResponsibilitySet:922 is extremely poorly modelled.

[skinkie]

It is likely that you over simplied the feed. Therefore I cannot recover the required properties for all lines.

<Line id="ch:1:slnid:1025132" version="any">
        <Name>Seblengrat</Name>
        <ShortName>Seblengrat</ShortName>
        <TransportMode>cableway</TransportMode>
        <TransportSubmode>
                <TelecabinSubmode>chairLift</TelecabinSubmode>
        </TransportSubmode>
        <PublicCode/>
</Line> 

[ue71603]

Open issues:

• Line oversimplification: What elements are you missing? OperatorRef will be added (/)
• Empty I will remove them (/)
• OperatorRef: Our realisation guide says, that it should be on the Line https://www.oev-info.ch/sites/default/files/2024-05/NeTEx_Core-Realisation_Guide_TP_Suisse-v1.00.pdf Is this where you miss it? (/)
• AvailibilityCondition : Yes they didn't do this right.. => will tell them
• Modelling of ch:1:ResponsibilitySet:922: Oh yes. They will have to change that too.

As I want to change them stuff when we are done with everything, manually change the things and attach the update netex.zip here: netex.zip

[skinkie]

The issues you mention are related to the source data. Hence we have now a way to infer OperatorRef from a ResponsibilitySet and apply an OperatorRef to a Line from referencing ServiceJourneys. But not yet via ServiceJourneyPattern -> Route -> Line. We are currently missing it at Line level.