skinkie / reference

Personal repository where I collect working examples to understand inner workings while building PyNeTExConv
GNU Affero General Public License v3.0
1 stars 1 forks source link

Trenitalia does not export the ServiceJourneyPatterns for some reason. #64

Closed ue71603 closed 2 months ago

ue71603 commented 3 months ago

file: IT-ITH3-TRENITALIA-NeTEx_L1.zip

commands

del D:/development/conversion/it_tren/*.duckdb
python netex_to_db.py D:/development/conversion/it_tren/IT-ITH3-TRENITALIA-NeTEx_L1.xml D:/development/conversion/it_tren/sncf_netex-import.duckdb
python epip_db_to_db.py D:/development/conversion/it_tren/sncf_netex-import.duckdb D:/development/conversion/it_tren/netex-import-epip.duckdb
python epip_db_to_xml.py D:/development/conversion/it_tren/sncf_netex-import.duckdb D:/development/conversion/it_tren/netex-import-epip.duckdb D:/development/conversion/it_tren/netex.xml
python netex_stats.py D:/development/conversion/it_tren/netex.xml
del D:/development/conversion/it_tren/*.duckdb

output from epip_db_to_db.py


(.venv) PS D:\development\github\reference\gtfs-netex-test> python epip_db_to_db.py D:/development/conversion/it_tren/sncf_netex-import.duckdb D:/development/conversion/it_tren/netex-import-epip.duckdb
CREATE TABLE IF NOT EXISTS AvailabilityCondition (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS DestinationDisplay (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS Direction (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS Line (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS Notice (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS NoticeAssignment (id varchar(64) NOT NULL, version varchar(64) NOT NULL, ordr integer, object text NOT NULL, PRIMARY KEY (id, version, ordr));
CREATE TABLE IF NOT EXISTS Operator (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS PassengerStopAssignment (id varchar(64) NOT NULL, version varchar(64) NOT NULL, ordr integer, object text NOT NULL, PRIMARY KEY (id, version, ordr));
CREATE TABLE IF NOT EXISTS RouteLink (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS RoutePoint (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS ScheduledStopPoint (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS ServiceJourney (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS ServiceJourneyPattern (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS StopPlace (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
CREATE TABLE IF NOT EXISTS VehicleType (id varchar(64) NOT NULL, version varchar(64) NOT NULL, object text NOT NULL, PRIMARY KEY (id, version));
epip_line_memory
Line 5
 ScheduledStopPoint 312

 ScheduledStopPoint 312epip_site_frame_memory
PassengerStopAssignment 312
 PassengerStopAssignment 312StopPlace 312
 StopPlace 312epip_service_journey_generator
 ServiceJourney 1771

ServiceCalendar 1
 ServiceJourneyPattern 0
ue71603 commented 2 months ago

any idea, why this is not working? For other italian files I guess more corrections are needed before (currently NAP not operational)

ue71603 commented 2 months ago

still no ServiceJourneyPattern. They exist in the original file

ue71603 commented 2 months ago

Update: The probelm is that there are ServiceJourneyPatternRef still in it. The information itself is stored in the PassingTimes. So perhaps you just have to remove the Ref to the SerivceJourneyPattern.

skinkie commented 2 months ago

PassingTimes always needs a ServiceJourneyPattern, because it does not have order nor ScheduledStopPoints (Calls do).

ue71603 commented 2 months ago

Then the output for SNCF is wrong, too. There are PassingTimes, but no ServiceJourneys.

skinkie commented 2 months ago

That statement is impossible ;) ServiceJourneyPatterns I guess?

ue71603 commented 2 months ago

IT

              <passingTimes>
                <TimetabledPassingTime id="IT:ITH3:TimetabledPassingTime:7-VE865-000-0083_vene_1" version="33">
                  <StopPointInJourneyPatternRef version="33" ref="IT:ITH3:StopPointInJourneyPattern:101_0"/>
                  <DepartureTime>07:10:00</DepartureTime>
                </TimetabledPassingTime>
                <TimetabledPassingTime id="IT:ITH3:TimetabledPassingTime:7-VE865-000-0083_vene_2" version="33">
                  <StopPointInJourneyPatternRef version="33" ref="IT:ITH3:StopPointInJourneyPattern:101_1"/>
                  <ArrivalTime>07:19:00</ArrivalTime>
                  <DepartureTime>07:19:00</DepartureTime>
                </TimetabledPassingTime>
              </passingTimes>

SNCF:


              <TimetabledPassingTime id="FR:TimetabledPassingTime::SN14140FERRE_1385103-1" version="any">
                  <PointInJourneyPatternRef ref="FR:StopPointInServiceJourneyPattern::87673004_SN14140FERRE_1385103"/>
                  <DepartureTime>06:59:00</DepartureTime>
                </TimetabledPassingTime>
                <TimetabledPassingTime id="FR:TimetabledPassingTime::SN14140FERRE_1385103-2" version="any">
                  <PointInJourneyPatternRef ref="FR:StopPointInServiceJourneyPattern::87672253_SN14140FERRE_1385103"/>
                  <ArrivalTime>07:44:00</ArrivalTime>
                  <DepartureTime>07:46:00</DepartureTime>
                </TimetabledPassingTime>

no id="FR:StopPointInServiceJourneyPattern::87673004_SN14140FERRE_1385103" but only reference

And also. After I "fixed" the Datasource problem. The result was valid. Even withouth the relsolved ref for the StopPointInServiceJourneyPattern ... we miss an id check there. netex.zip

skinkie commented 2 months ago
  1. Missing ServiceJourneyPatterns (because they existed in the source and therefore it is not in target_db, it is in the source_db)
  2. version attribute missing on many elements
  3. TimetabledPassingTime/PointInJourneyPatternRef is this valid according to EPIP (or their profile). We are missing validation rules for PointInJourneyPatternref in NeTEx_publication.xsd
skinkie commented 2 months ago

Fixed.