SensorElementList should only appear once

[RalphTro]

Dear All,

When I executed the file CertificationInfoAndSensorReportWithCRS.jsonld , the field sensorElementList appeared twice in the pre-hash string:

...
sensorElementList
sensorElement
sensorMetadata
time=2023-01-11T11:05:00.000Z
bizRules=https://id.gs1.org/253/4012345000054987
sensorReport
type=https://gs1.org/voc/Dimensionless
exception=https://gs1.org/voc/ALARM_CONDITION
coordinateReferenceSystem=http://www.opengis.net/def/crs/EPSG/0/27700
sensorElementList
sensorElement
sensorMetadata
...

Similar to e.g. bizTransactionList, this field should only appear once.

Kind regards, @RalphTro

[dakbhavesh]

Dear @RalphTro, I checked with Aravinda about how pre-hashes are produced in Java and JavaScript based implementation and got to know that they are also creating separate sensorElementList only for enclosed user extension fields. However, if you think that it's not the right way to generate pre-hash then we will surely fix the logic.

OpenEPCIS Hash Generator Link: https://tools.openepcis.io/openepcis-ui/EventHash/

Our suggestion is to enrich CBV standard so that we all are on the same page about the expected pre-hash structure for all important permutations & combinations.

Looking forward to your confirmation.

Thanks, Bhavesh

[RalphTro]

Dear @dakbhavesh , Thanks a lot for checking this! As to your suggestion: happy to discuss to which extent it makes sense to make our specification more clear. I think an appropriate place for adding some language would be:

11. All child elements as part of a list (e.g. epc in epcList, bizTransaction in bizTransactionList, etc.) SHALL be sequenced according to their case-sensitive lexical ordering, considering UTF-8/ASCII code values of each successive character.

This may not be explicit enough that the parent field should only appear once. From your POV, which additional language would make sense? E.g. sth. like "The name of a field denoting a list (e.g. epcList, bizTransactionList, sensorElementList) SHALL only appear once."? Please go ahead and suggest any modification which would make sense from your perspective.

Kind regards, Ralph

[dakbhavesh]

Dear @RalphTro, In my opinion, the definition/language in point#11 looks fine however we should rather update the canonical ordering of elements as we are missing mention of the user extension field there.

For example,

As is:

23. sensorElement (
sensorMetadata (time, startTime, endTime, deviceID, deviceMetadata, rawData, dataProcessingMethod, bizRules),
sensorReport (type, deviceID, deviceMetadata, rawData, dataProcessingMethod, time, microorganism,
chemicalSubstance, value, component, stringValue, booleanValue, hexBinaryValue, uriValue, minValue, maxValue,
meanValue, sDev, percRank, percValue, uom))

To be:

23. sensorElement (
sensorMetadata (time, startTime, endTime, deviceID, deviceMetadata, rawData, dataProcessingMethod, bizRules, userExtensions),
sensorReport (type, deviceID, deviceMetadata, rawData, dataProcessingMethod, time, microorganism,
chemicalSubstance, value, component, stringValue, booleanValue, hexBinaryValue, uriValue, minValue, maxValue,
meanValue, sDev, percRank, percValue, uom))

What do you think?

[dakbhavesh]

Dear @RalphTro, Please disregard my previous comment as there are better places where we can incorporate more clarity in language and sufficient examples.

• For language, we should modify the rule point-20 as follows

As is:

If an EPCIS event comprises user extension elements as part of an EPCIS standard field with an extension point
(namely readPoint, bizLocation, sensorElement, sensorMetadata, and sensorReport), the top-level user extension element(s) SHALL be prefixed with
the corresponding EPCIS standard field name. Apart from that, it SHALL be added to the pre-hash string similarly as specified in the previous step.

To be:

If an EPCIS event comprises user extension elements as part of an EPCIS standard field with an extension point
(namely readPoint, bizLocation, sensorElement, sensorMetadata, and sensorReport), the user extension should be added at the end of its enclosing parent’s regular fields. Apart from that, pre-hash SHALL be prepared similarly as specified in the previous step.

[@RalphTro - Above is just a suggestion as to what kind of phrasing will make it clear however you are far better at coming up with generally understandable phrasing]

• For examples, we should modify point-25 which is under "Canonical Property Order" to include user extension for each field with max complexity where it is allowed i.e. readPoint, bizLocation, sensorElement, sensorMetadata, and sensorReport. This will make it absolutely clear how a pre-hash string will look for a comprehensive example.

Please let me know if the above makes sense, Bhavesh

[RalphTro]

@dakbhavesh: Thanks for sharing your thoughts as to the wording! I suggest that I will set up an e-mail to Craig about this matter and copy you in (no need to discuss this via GitHub in my view).

[RalphTro]

When digging into this matter next week (when you have the bandwidth), please note that even though sensorElementList should not appear several times, sensorElement should (of course, if there are multiple contained sensorElements :-))

[RalphTro]

Just checked epcisDocWithSensorDataObjectEvent.jsonld in the masterbranch.

Im my opinion, the sequence looks as it should be (removed all irrelevant fields):

sensorElementList

sensorElement
sensorMetadata
sensorReport
sensorReport
sensorReport
sensorReport

sensorElement
sensorMetadata
sensorReport
sensorReport
sensorReport
sensorReport

sensorElement
sensorMetadata
sensorReport
sensorReport
sensorReport
sensorReport

So, I think we can close this issue tomorrow.

[RalphTro]

Related issue: what happens if sensorElementList contains user extensions? Bhavesh to have a look - THANKS!

[RalphTro]

Dear @dakbhavesh , marvellous! I think we are almost there. The only thing I noticed is the following: for testing purposes, I created an EPCIS event with user extensions in all fields with an extension point, and also included 2 examples with more than one user extension:

{
    "@context": [
        "https://ref.gs1.org/standards/epcis/2.0.0/epcis-context.jsonld",
        {
            "cbv": "https://ref.gs1.org/cbv/"
        },
        {
            "ext1": "https://ns.ext1.de/epcis/"
        },
        {
            "ext2": "https://ns.ext2.de/epcis/"
        }
    ],
    "type": "EPCISDocument",
    "schemaVersion": "2.0",
    "creationDate": "2023-01-11T12:00:00.000+01:00",
    "epcisBody": {
        "eventList": [
            {
                "type": "ObjectEvent",
                "eventTime": "2023-01-11T12:00:00+01:00",
                "eventTimeZoneOffset": "+01:00",
                "epcList": [
                    "https://id.gs1.de/01/04012345999990/21/XYZ-1234"
                ],
                "action": "OBSERVE",
                "bizStep": "receiving",
                "disposition": "in_progress",
                "readPoint": {
                    "id": "https://id.gs1.de/414/4012345000115",
                    "ext1:readPointField": "readPointText",
                    "ext1:readPointField2": "readPointText2"
                },
                "bizLocation": {
                    "id": "https://id.gs1.de/414/4012345000115",
                    "ext1:bizLocationField": "bizLocationText"
                },
                "sensorElementList": [
                    {
                        "sensorMetadata": {
                            "time": "2023-01-11T11:05:00Z",
                            "bizRules": "https://id.example.com/253/4012345000054987",
                            "ext1:someFurtherMetadata": "metadataText",
                            "ext1:someFurtherMetadata2": "metadataText2"
                        },
                        "sensorReport": [
                            {
                                "type": "Dimensionless",
                                "exception": "ALARM_CONDITION",
                                "ext1:someFurtherReportData": "reportText"
                            }
                        ],
                        "ext1:someSensorElementField": "sensorElementText"
                    },
                    {
                        "sensorReport": [
                            {
                                "type": "gs1:Angle",
                                "value": 42.698334,
                                "component": "cbv:Comp-latitude",
                                "uom": "DD",
                                "coordinateReferenceSystem": "http://www.opengis.net/def/crs/OGC/1.3/CRS84",
                                "ext2:secondSensorReportField": "secondSensorReportText"
                            },
                            {
                                "type": "gs1:Angle",
                                "value": 23.319941,
                                "component": "cbv:Comp-longitude",
                                "uom": "DD",
                                "coordinateReferenceSystem": "http://www.opengis.net/def/crs/OGC/1.3/CRS84"
                            }
                        ],
                         "ext2:secondSensorElementField": "secondSensorElementText"
                    }
                ]
            }
        ]
    }
}

The algorithm already works quite good if there is only one user extension - the only thing what needs to be adjusted is that the second user extension fields in e.g. readPoint should be handled consistently to how it is the case with the other ones. As of now, they appear at the end of the pre-hash string:

...
{https://ns.ext2.de/epcis/}secondSensorElementField=secondSensorElementText
readPoint{https://ns.ext1.de/epcis/}readPointField2=readPointText2
sensorElementListsensorElementsensorMetadata{https://ns.ext1.de/epcis/}someFurtherMetadata2=metadataText2

Once this is adjusted, I think we should be fine.

Many thanks in advance for looking into this!

[dakbhavesh]

Thanks, @RalphTro for your quick response. I will look into your observation and fix it as soon as I can.

[dakbhavesh]

Dear @RalphTro, I have fixed a glitch in logic. Can you please try running your tests again and see if it's looking better now?

Thanks, Bhavesh

[RalphTro]

Many thanks, @dakbhavesh!

Taking the above-mentioned example, the user extensions under readPoint/bizLocation now look correct, e.g.:

readPointid=https://id.gs1.org/414/4012345000115
{https://ns.ext1.de/epcis/}readPointField2=readPointText2
{https://ns.ext1.de/epcis/}readPointField=readPointText

Also, the user extensions under sensorElement is correctly appended to the pre-hash string.

The same holds true for the user extensions in sensorMetadata and sensorReport.

I tried it out both on Mac and Windows.

So, all looks good now! GREAT! :-)

[RalphTro]

Can be closed as of latest PR.