skinkie / reference

Personal repository where I collect working examples to understand inner workings while building PyNeTExConv
GNU Affero General Public License v3.0
0 stars 1 forks source link

Issue with Swiss data #65

Closed ue71603 closed 1 month ago

ue71603 commented 1 month ago

Question:

                <StopPointInJourneyPattern id="OPENOV:StopPointInJourneyPattern:bee9eb39-1" version="any" order="1">
                  <ScheduledStopPointRef ref="ch:1:ScheduledStopPoint:8500010:12" versionRef="any"/>
                  <noticeAssignments>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_1089-001_9103WY.j24_16_1" version="any" order="1">
                      <NoticeRef ref="ch:1:Notice:A__BA" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_1089-001_9103WY.j24_16_1" version="any" order="2">
                      <NoticeRef ref="ch:1:Notice:A__WR" versionRef="any"/>
                    </NoticeAssignment>
                  </noticeAssignments>

is this valid or not? same id, but different order.


(.venv) PS D:\development\github\reference\gtfs-netex-test> python netex_to_db.py D:\development/conversion/swiss/netex.xml  D:\development/conversion/swiss/netex-database.duckdb

CREATE TABLE IF NOT EXISTS ZoneProjection (id varchar(64) NOT NULL, version varchar(64) NOT NULL, ordr integer, object text NOT NULL, PRIMARY KEY (id, version));    
<NoticeAssignment xmlns="http://www.netex.org.uk/netex" xmlns:gml="http://www.opengis.net/gml/3.2" id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_1089-001_9103WY.j24_16_1" version="any" order="2"><NoticeRef ref="ch:1:Notice:A__WR" versionRef="any"/></NoticeAssignment>
Traceback (most recent call last):
  File "D:\development\github\reference\gtfs-netex-test\netex_to_db.py", line 33, in <module>
    main(args.netex, args.database, args.clean_database)
  File "D:\development\github\reference\gtfs-netex-test\netex_to_db.py", line 15, in main
    with sqlite3.connect(database) as con:
  File "D:\development\github\reference\gtfs-netex-test\netex_to_db.py", line 23, in main
    insert_database(con, classes, sub_file)
  File "D:\development\github\reference\gtfs-netex-test\netexio\dbaccess.py", line 473, in insert_database
    cur.execute(sql_insert_object, (id, version, order, xml,))
duckdb.duckdb.ConstraintException: Constraint Error: Duplicate key "id: ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_1089-001_9103WY.j24_16_1, version: any" violates primary key constraint. If this is an unexpected constraint violation please double check with the known index limitations section in our documentation (https://duckdb.org/docs/sql/indexes).
(.venv) PS D:\development\github\reference\gtfs-netex-test>
skinkie commented 1 month ago

Yes, that is why I required to get TimetabledPassingTime with order :-) So they really did a id+version+oder as unique element, also for other element. But the problem in this case is that order has been made optional thus it instantly became impossible to import after I fixed optional elements for SNCF...

skinkie commented 1 month ago
<xsd:unique name="NoticeAssignment_UniqueBy_Id_Version_Order">
        <xsd:annotation>
                <xsd:documentation>Every [NoticeAssignment Id + Version +order] must be unique within document.</xsd:documentation>
        </xsd:annotation>
        <xsd:selector xpath=".//netex:NoticeAssignment"/>
        <xsd:field xpath="@id"/>
        <xsd:field xpath="@version"/>
        <xsd:field xpath="@order"/>
</xsd:unique>

I think the schema made order optional...

skinkie commented 1 month ago

NoticeAssignment should not have been imported, because it was part of another object.

ue71603 commented 1 month ago

Generated objects always should have unique id => to be discussed with MENTZ

skinkie commented 1 month ago

As discussed, this all uncovered some fundamental problems as well.

ue71603 commented 1 month ago

we close it here.