problem with related explorer

[ue71603]

I tested it with Swiss4 swiss4-netex.zip

command (in script: ` {"script": "related_explorer.py", "args": "%%dir%%/netex-database.duckdb ServiceJourney random %%dir%%/sj.xml"},

`

related_explorer.py with d:/aux_testing_processing/swiss4/netex-database.duckdb Line random d:/aux_testing_processing/swiss4/line.xml

fixed version: sj.zip

Problems:

• DayType was not added
• Operator was duplicated
• Line was duplicated

Then it was valid

[ue71603]

I also tested this sequence (for a random Line) {"script": "related_explorer.py", "args": "%%dir%%/netex-database.duckdb Line random %%dir%%/line.xml"}

or fully: related_explorer.py with d:/aux_testing_processing/swiss4/netex-database.duckdb Line random d:/aux_testing_processing/swiss4/line.xml

It did not add any ServiceJourneys. @skinkie is this expected?

result was:

<?xml version="1.0" encoding="UTF-8"?>
<PublicationDelivery xmlns="http://www.netex.org.uk/netex" xmlns:gml="http://www.opengis.net/gml/3.2" version="ntx:1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.netex.org.uk/netex file:///C:/Users/ue71603/MG_Daten/github/NeTEx1/xsd/NeTEx_publication.xsd">
    <PublicationTimestamp>2024-10-05T20:40:11.224002</PublicationTimestamp>
    <ParticipantRef>PyNeTExConv</ParticipantRef>
    <dataObjects>
        <GeneralFrame id="Results" version="1">
            <members>
                <Line id="ch:2:Line:72.EXT.1" version="any" responsibilitySetRef="ch:1:ResponsibilitySet:RhB_72_RhB">
                    <ValidBetween>
                        <FromDate>2023-12-10T00:00:00</FromDate>
                        <ToDate>2024-12-14T23:59:59</ToDate>
                    </ValidBetween>
                    <Name lang="de">1</Name>
                    <ShortName lang="de">1</ShortName>
                    <TransportMode>rail</TransportMode>
                    <TransportSubmode>
                        <RailSubmode>specialTrain</RailSubmode>
                    </TransportSubmode>
                    <PublicCode>1</PublicCode>
                    <OperatorRef nameOfRefClass="Operator" ref="ch:1:Operator:72"/>
                </Line>
                <Operator id="ch:1:Operator:72" version="any">
                    <PrivateCode>72</PrivateCode>
                    <Name>Rhätische Bahn</Name>
                    <ShortName>RhB</ShortName>
                </Operator>
            </members>
        </GeneralFrame>
    </dataObjects>
</PublicationDelivery>

[ue71603]

@skinkie beside Line and ServiceJourney: What else should the related_explorer be able to scan?

And how to we make sure it doesn't start copying to much into the file?

[skinkie]

As mentioned in our conversation:

1. https://github.com/skinkie/reference/issues/43
2. I think a strategy could be do not include new objects of the same type, but I can think of use cases where this might be stupid.
3. A file will not be valid if it references objects that are not present. So ServiceJourney -> BlockRef -> Block -> ServiceJourneyRef -> ServiceJourney will pull in those other trips, and potentially lines. Maybe an option stating "Exclude object type" might help.

I thought I already resolved the duplicate objects, but evidently not in the repository.

You asked if it is expected that no ServiceJourney is added. Yes, for now this is expected, because no inward relation (#43) is resolved. So basically an object with LineRef will not cause it to be added (yet).

[ue71603]

did it from the Swiss basic... missed this element: <TypeOfProductCategory id="ch:1:TypeOfProductCategory:R" version="any"/> <!-- was added manually --> sj2.zip

[skinkie]

@ue71603 is swiss4-netex basic?

[skinkie]

https://github.com/skinkie/reference/commit/265c4462825b533e22219105b44b6fd5c71ab354 takes care of the duplicates.

The DayType is more difficult. The DayType definition is part of the ServiceCalendar (it is an embedded object). We need to have a way to know which objects are part of other objects (for example a Quay in a StopPlace).

[skinkie]

@ue71603 I am very unhappy with these duplicate ids, and yes I know they are currently allowed:

                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="1">
                      <NoticeRef ref="ch:1:Notice:A__FS" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="2">
                      <NoticeRef ref="ch:1:Notice:A__BA" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="3">
                      <NoticeRef ref="ch:1:Notice:A__TT" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="4">
                      <NoticeRef ref="ch:1:Notice:A__WR" versionRef="any"/>
                    </NoticeAssignment>

[skinkie]

https://github.com/skinkie/reference/commit/85c8d65e5319611281be57451d9b9da90aefae24

Will resolve the correct embeddings. Requires a new import of the contents.

[ue71603]

@skinkie

@ue71603 I am very unhappy with these duplicate ids, and yes I know they are currently allowed:

                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="1">
                      <NoticeRef ref="ch:1:Notice:A__FS" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="2">
                      <NoticeRef ref="ch:1:Notice:A__BA" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="3">
                      <NoticeRef ref="ch:1:Notice:A__TT" versionRef="any"/>
                    </NoticeAssignment>
                    <NoticeAssignment id="ch:1:NoticeAssignment:ch_1_ServiceJourney_ch_1_sjyid_100001_10536-002_91005A.j24_1872_1" version="any" order="4">
                      <NoticeRef ref="ch:1:Notice:A__WR" versionRef="any"/>
                    </NoticeAssignment>

so we want them to have different id? I mentioned it here: https://github.com/MMTIS/QA/issues/4

[ue71603]

@ue71603 is swiss4-netex basic?

What do you mean by basic? It is a part that I extracted so that we can study the JourneyMettings as well. It si all other frames + some timetableframes

[skinkie]

What do you mean by basic? It is a part that I extracted so that we can study the JourneyMettings as well. It si all other frames + some timetableframes

I did not understand what you ment here:

did it from the Swiss basic...

so we want them to have different id?

I think we also must change this from the documentation and in the upstream XML Schema. Having unique ids would be something really good.

[skinkie]

@ue71603 please validate that related explorer works for you. Use netex_to_db.py --referencing to create the extra table. I think it would be very interesting if we can use this to export a GeneralFrame, import it, and export a valid EPIP file (for testing).