PC Rules referencing attributes or attribute values that have been discontinued

[alvarosanuy]

As an example, the Vegetation.lua rule refers to a categoryOfVegetation for Mangroves, that has now been discontinued and removed from the FC. Although the portrayal instruction should not be triggered as the attribute value is prohibited within an ENC, it seems appropriate keep the PC and FC in line.

Please provide your views and recommendations moving forward.

Comment from @mikan66: the invalid attribution for specific rules, this is the result of changing feature catalogs most likely. We'll need to decide how to clean this up as well. It will most likely be a manual process on a feature-by-feature basis from the catalog as checked against each rule file for the final release. With 200+ rules this would take some time. For now, maybe case-by-case as found would be better?

[dnwkfkwk]

What if we verify whether all features and attributes used in the PC are present in the FC at this point in time, and then create a list of modifications and resolve them one by one? Could this be a feasible approach?

[TomRichardson6]

A question I have here is; whether once we enter the Operational period (2.0.0 onwards) the Portrayal Catalogue should retain deprecated rules and symbols so that for example Portrayal Catalogue 2.1 also supports data that complies with FC 2.0? It is clearly impractical to issue new editions of all ENC datasets when a new Portrayal Catalogue is issues so I think this is a means to achieve backwards compatibility. Support removal of disused content at this stage as this doesn't apply but I think we should maintain this for specific FC versions until they become fully unsupported in that timeframe all datasets will then need to be reissued to reflect this.

[DavidGrant-NIWC]

Older datasets continue to use the rules in the older PC. A new FC requires new data and a new PC.

[skjeves]

Hi guys,

I am preparing a paper related to dataset vs catalogue referencing for the upcoming S-100 meeting. Not sure to how large degree it is related to your discussion here, but I think there may be commonalities. If you would have a look it would be much appreciated. Do you agree this relationship is not covered yet in S-100 5.1.0, or have we missed something? Catalogue and dataset versioning_draft version .docx

[TomRichardson6]

@skjeves thanks for progressing this we didn't get time for my versioning paper at PT11 which also touches on this. I agree that a dataset must reference a single FC. But for Portrayal Catalogues I think this constraint is counter productive and limits the machine readability concept. For example introduction of a new symbol will take many years to reach all cells (many ENCs have not had new editions for 10 years) and reissuing data creates nugatory effort. Allowing Portrayal Catalogues to work with multiple versions of a feature catalogue offers greater flexibility and ensures that data presents consistently. Some rules would need to be in place for this but this works for AML for example as that Presentation Library supports multiple editions and Feature Catalogues.

Whatever route is chosen S-98 needs to be more clear on this. So I will try and review the paper in detail.

[kusala9]

I think a general rule for backwards compatibility isn't well described anywhere - it would need documenting in S-98 (and I believe there is a starting point there already). NIWC have raised this on a number of occasions and there's an issue for it in S-164 already as well. I think the general rule is (although I'm not sure this is properly written down anywhere (yet)):

FC with version X.Y.Z+1 should also be backward compatible with X.Y.Z 100% but it becomes more murky when X.Y+1.* is defined.

I would definitely welcome this being better defined and will review paper above. S100WG would be a good opportunity to get it sorted....

[kusala9]

ah, I pressed send at the same time as @TomRichardson6 :-).. Agree needs better defn (and testing in S-164 - we have tests for multiple FC/Dataset versions but could use some work. v1.2 gives us something concrete to work with.

[DavidGrant-NIWC]

I'm all for better requirements, but a PC should not target multiple FC's. The PC can be updated at any time to change symbology/colors/etc. - it doesn't require an updated dataset or FC. However, when the FC is updated the PC and data must also be updated. Depending on the scope of the changes, the older datasets can be brought into alignment by releasing an update to the older PC in parallel with pushing the latest revision.

Exchange Set	Datasets	FC	PC	Notes
1	-	1.0.0	1.0.0	Initial delivery of FC and PC
2	A, B	-	-	Initial delivery of datasets compiled using FC 1.0.0
3	A	1.1.0	1.1.0	Update dataset A, FC, and PC
4	-	-	1.0.1	Update older PC to match current (as best as possible); will update portrayal of dataset B which is still on FC 1.0.0

@skjeves, it isn't necessary to add featureCatalogueVersion and portrayalCatalogueVersion to S100_ProductSpecification. S100_CatalogueDiscoveryMetadata already provides productSpecification and we had intended the S100_ProductSpecification productIdentifier field to uniquely identify a version of a product spec. The attribute could be used to tie the catalogs and datasets together, but the registry has yet to be updated to support this, and there is also no supporting test data; we had to proceed with an alternate implementation so that we could move on with testing.

We currently link datasets to FC's via the DSID/PRSP subfield in the 8211 encoding. In a GML encoding this would be the productIdentifier field (Table 10b-4), and in HDF-5 you could look at productSpecification (Table 10c-6). My understanding is that the FC version is supposed to align with the Product Spec version, although I don't think there is currently a written requirement we can reference. We establish a match when the first two version elements match. So, INT.IHO.S-101.1.1 matches FC 1.1.x, INT.IHO.S-101.1.2 matches FC 1.2.x, etc.

The ECDIS doesn't really care what the version of the product specification is. It just wants to associate each dataset with a FC, and each PC with a FC. The association of a dataset to a PC is indirect (through the FC). Our preference would be:

• Assign an MRN to each FC by adding an element to the FC. <S100FC:mrn>urn:mrn:iho:fc:...</S100FC:mrn>
• Provide a required field in each encoding which stores the MRN of the FC used to produce the data.
• The exchange set creator could copy the MRN from the encoded dataset to a field in the XS discovery metadata.
  • Minimizes risk of human error
• Each PC would reference its associated FC via MRN: <portrayalCatalog productId="S-101" version="whatever" fc="urn:mrn:..."/>

Unfortunately, I think it's too late to make this change in S-100 v5. Since v5 will be used for production, there will be no point in making the change in v6.

[skjeves]

@DavidGrant-NIWC I think your table demonstrates well how you envisage to update older PCs to match new ones - it should be proposed. If this becomes an agreed upon mechanism it must be described - is S-98 the correct place for that?

This is the table describing different versioning use cases:

If I understand correctly the association between dataset and PC is not relevant - this is an indirect relationship defined in the FC. What happens then when you have an uptick in PC version without uptick in FC? (use case 4 and 8 in the table)

"it isn't necessary to add featureCatalogueVersion and portrayalCatalogueVersion to S100_ProductSpecification. S100_CatalogueDiscoveryMetadata already provides productSpecification and we had intended the S100_ProductSpecification productIdentifier field to uniquely identify a version of a product spec. The attribute could be used to tie the catalogs and datasets together, but the registry has yet to be updated to support this, and there is also no supporting test data; we had to proceed with an alternate implementation so that we could move on with testing". For use cases where catalogues can be updated independently of the product specification, this may work with the caveat (defined in S-98 (?)) that the newest catalogue must be used. S100_CatalogueDiscoveryMetadata - S100_ProductSpecification - productIdentifier field would then be equal in different versions of the catalogue. (use case 2, 3, 4, 7 and 8 in the table).

"We currently link datasets to FC's via the DSID/PRSP subfield in the 8211 encoding. In a GML encoding this would be the productIdentifier field (Table 10b-4), and in HDF-5 you could look at productSpecification (Table 10c-6). My understanding is that the FC version is supposed to align with the Product Spec version, although I don't think there is currently a written requirement we can reference. We establish a match when the first two version elements match. So, INT.IHO.S-101.1.1 matches FC 1.1.x, INT.IHO.S-101.1.2 matches FC 1.2.x, etc" I think your understanding of alignment between FC version and ProdSpec version is not correct. (use case 2,3 and 7 in the table).

If your preferred solution is assigning MRNs to FC and dataset encodings, I think you should propose it nevertheless. We have made other suggestions that has been determined not suitable before S-100 v6 - and although v5 will be used for production I guess continous development of the S-100 framework will take place eventually (especially based on all the topics still to be uncovered when people start implementing support for v5).

[alvarosanuy]

Very good discussions (added 'New S-164 Test' and 'S-98 Change Proposal' labels) but I think the conversation deviated a bit from the original intent.

I think that, as a starting point, we should aim at releasing a PC 2.0.0 that does not contain references to features, attributes, geometries combinations that are not supported by FC 2.0.0.

The rest of the conversation is about S-100 Catalogues' management and operationalization in ECDIS moving forward. Probably something to discuss at the S100WG level ... I understand the Catalogues management bit will be discussed at S100WG8 through a PRIMAR paper anyway. Correct?

• [ ] Recommend this issue is only used to resolve how to clean up the PC content and align it with the corresponding FC version (aiming at full synchronisation for 2.0.0).

[skjeves]

@alvarosanuy Sorry for intruding your initial discussion in this issue. Yes I will provide a paper to the upcoming S100WG meeting.

I have created a new issue at the S100WG repository - catalogue vs dataset versioning discussion can continue there. @TomRichardson6 @kusala9 @DavidGrant-NIWC here is the link to the new issue: https://github.com/iho-ohi/S-100WG/issues/10

[alvarosanuy]

Decisions made at Portrayal subWG meeting on 19/10/23

It was agreed that, although the mapping of deprecated attributes is not visible to the user, it's not a good practice and can confuse some data producers or other stakeholders that look at the Rules to understand how encoding impacts portrayal on ECDIS. If deprecated attributes or attribute values are retained in the syntaxes of a Rule, readers may waste their time going back and forth to the DCEG and the FC trying to confirm what's right and what's wrong.

• [ ] NIWC to do their best to stop this practice from continuing and consider cleaning up deprecated attributes or attribute values from all S101 PC rules in PC 2.0.0.

[alvarosanuy]

Decisions made at Portrayal subWG meeting on 09/04/24

Cost-benefit of this activity is very low.

Close this issue and recommend NIWC opens an Issue in their space to keep track of this requirement (post 2.0.0 release and ideally before PC Rules start to be managed by the IHO GI Registry - PC Builder (?))