S-101-Portrayal-subWG / Working-Documents

17 stars 5 forks source link

Quality of Bathymetric Data - Visualization of Vertical and Positional Uncertainties #1

Closed alvarosanuy closed 1 year ago

alvarosanuy commented 4 years ago

S-101PT5 Action 03 - Consider the implementation of DQWG recommendations on improved portrayal of Quality of Bathymetric Data information in ECDIS.

DQWG15_2020_05.1B_EN_Uncertainty of measurements and ECDIS performance.docx - original proposal

S-101PT5_13_EN_S-100WG5_2020_3.8_EN_DQWG_Report_Paper.pdf - refer to item 3

HSSC12_2020_05.5C_EN_Conversion of M_QUAL-CATZOC to S-101 (1).pptx

The subWG is invited to:

  1. Approve the use of Horizontal Position Uncertainty & Vertical Uncertainty to affect portrayal and ECDIS performance as described in the paper.
  2. Approve the S-57 to S-101 migration strategy proposed by the DQWG and refer it to the relevant ENCWG focus group for consideration.
  3. Approve the DQWG proposal of adding a new boolean attribute (uncertaintyDisplay Yes/No) to control the display of uncertainties indicators in ECDIS to the following feature types: Sounding, Underwater Rock, Wreck and Obstruction.
  4. Consider the merits of applying a similar approach to bridges and pipelines/cables overhead using the uncertainties populated in the complex attributes Vertical Clearance Fixed and Horizontal Clearance Fixed.
  5. If approved, come up with a way forward to develop appropriate symbology and ECDIS functionalities for testing the principles outlined in the proposal (NIWC/KOHA ??). As a minimum, agree on recommending the S101PT on updating the FC and including an entry to the ‘S-100 ECDIS Performance requirements log’ for post DF implementation.
  6. Consider the merits of accepting the DQWG’s suggestion of removing categoryOfTemporalVariation from QoBD. – Note that this request conflicts with the proposal outlined in Issue # 4.
alvarosanuy commented 4 years ago

The AHO:

Please notice that the following comment made in the DQWG paper, and which is repeated for Wrecks, Sounding and Underwater Rocks, is not accurate. 'The Obstruction Feature type is missing the attribute ref: horizontalPositionUncertainty, which is needed for the permittedPrimitive = point.' horizontalPositionUncertainty is not missing from these feature types. It is a complex attribute of the Information type Spatial Quality. Each instance of Spatial Quality must be associated to the geometry to which the information applies using the association 'Spatial Association' (see DCEG clause 25.13).'

Christian-Shom commented 4 years ago

France (Shom):

I have one general question: If I understand correctly, we plan to have a S-101 portrayal that "looks like" S-52 during DF period (due to the need for an equivalent regime of presentation), and then improve (significantly ?) the S-101 portrayal when there are no more S-57 ENCs. Do we have documentation on this approach with a list of things we can do now and others we plan for later? Do we have a time table?

NLHO-Rogier commented 4 years ago

AT DQWG15 meeting UK rejected the existence of category of temporal variation in S-101. Their argument was that if a highly accurate survey is done, it should receive the highest CATZOC value. If significant shoaling is likely and would affect safe navigation, then the CATZOC value would immediately be downgraded. For that argument, the existing attribute values of category of temporal variation are not sufficient. However, downgrading of CATZOC over time should be possible where an HO finds this appropriate. DQWG could not come to a common agreement on this at the meeting and the issue is to be further discussed within this subWG.

SylviaSpohn-BSH commented 4 years ago

Good question from Christian.

The possibility of the temporal variation should be kept, but not mandatory. This is difficult to handle for us as we get every day a lot of little survey puzzles and for the Wadden Sea a monthly changing value does not fit (agree to Rogier, last comment).

alvarosanuy commented 3 years ago

Summary:

1 - Approved 2 - Christian M. (S-57 to S101 conversion focus group) to confirm 3 - Approved and implemented in DCEG 1.0.1 4 - Pending 5 - Added entry in ‘S-100 ECDIS Performance requirements log’ for post DF implementation. Will require creation of focus group as soon as S-101 2.0.0 is published. Mariners' input is critical. Recommend S-101PT Chair to log task with Singapore Laboratory. 6 - It has been decided to retain categoryOfTemporalVariation. For temporal variation discussions refer to https://github.com/S-101-Portrayal-subWG/Working-Documents/issues/4

DavidGrant-NIWC commented 3 years ago
alvarosanuy commented 2 years ago

Decisions and Actions at 01DEC2021 (post 2nd meeting in November 2021):

image

alvarosanuy commented 2 years ago

The focus of this Github issue is now to propose possible ways to visualize verticalUncertainty and horizontalPositionUncertainty in ECDIS for those features (Sounding, Underwater/Awash Rock, Wreck and Obstruction) having displayUncertainties = Yes.

The attribute displayUncertainties is a cartographic attribute intended to reduce screen clutter in some ECDIS display settings by limiting the display of the horizontal position and vertical accuracies to those considered by the encoder to be important to the mariner. This attribute only applies to Point geometry and is mandatory for affected features in depths 30 metres or less. Factors to be considered in populating this attribute include depth in relation to the general nature of the seabed, proximity to other dangers, intention of the ENC, proximity to routes taken by vessels, and the types of vessels intended to utilise the ENC.

Another element to discuss is the possibility of amending the ECDIS safety functions to enable the use of the 'modified' feature values (position/depth) when calculating safe passage. The attributes verticalUncertainty and horizontalPositionUncertainty would be used to create a buffer around the charted object and therefore, during route planning or route monitoring, the safety corridor would react to an area object (from a positional point of view), and to a shoaler depth.

SylviaSpohn-BSH commented 2 years ago

I suppose that an extra symbol for the basic ENC is not necessary. The value should be evaluated for route planning (safety distance shown from the object) As the values are not available comprehensively, we could develop the portyal later, not at a high priority.

alvarosanuy commented 2 years ago

Discussion topics:

forodd commented 2 years ago

First attempt at some testdata for horisontal and vertical uncertainty. This has been created by populating POSACC and SOUACC on individual objects in S-57 before converting to S-101 using FC 1.0.2.

Please let me know if this data need some adjustments, happy to update the data if needed.

Edit; data updated with dates for the QoBD features. Edit2: added attributs for dateStart to the QoBD features.

hor_ver_uncert_testdata_NO_v3.zip

DavidGrant-NIWC commented 2 years ago

image

kusala9 commented 2 years ago

happy to help with this one - apologies we haven't been able to prepare much data in support of the efforts here but we're doing some of the S-101 test data and can prepare test datasets for specific, simple cells. We have a YAML processor which produces S-101 conformant data. Attached cell has a QualityOfBathymetricData feature which works in v1.9.0 of the NIWC viewer. This is generated from the following YAML. If you want this modifying, let me know and I can produce a new version with the required attributes.....

  - Name: QualityOfBathymetricData
    Foid: 1810:1411:100
    Prim: Surface
    Attributes:
      - Name: categoryOfTemporalVariation
        Value: Unassessed
      - Name: dataAssessment
        Value: Unassessed
      - Name: featuresDetected
        id: 1
      - Name: significantFeaturesDetected
        parent: 1
        Value: false
      - Name: leastDepthOfDetectedFeaturesMeasured
        parent: 1
        Value: false
      - Name: surveyDateRange
        id: 2
      - Name: dateEnd
        parent: 2
        Value: 2021-01-01
      - Name: zoneOfConfidence
        id: 3
      - Name: categoryOfZoneOfConfidenceInData
        Value: Zone of Confidence U
      - Name: fullSeafloorCoverageAchieved
        Value: false

101USTESTOR001.zip

alvarosanuy commented 2 years ago

Decision made at Portrayal subWG meeting on 13/7/22

  1. Use test datasets to progress with testing of portrayal and A&I (see below). Leave the issue open.
  2. NIWC to implement the portrayal of horizontalPositionalUncertainty (value to be used should be the result of adding uncertaintyVariableFactor to uncertaintyFixed. If not possible use fixed value and report limitation or recommend new ECDIS display performance requirement to be passed to the S100WG) when displayUncertainties=TRUE. Applies to Sounding, Underwater/Awash Rock, Wreck and Obstruction features. Use the following line style:
  1. Request S100WG to develop a new ECDIS performance requirement to allow (at mariner's request) the triggering of A&I using 'worst case' scenarios for the position and depth of Sounding, Underwater/Awash Rock, Wreck and Obstruction features. This will be possible when verticalUncertainty and horizontalPositionalUncertainty values are populated (in SpatialQuality [when linked at the Point feature level] or in the categoryOfZoneInConfidenceInData of the overlying QoBD feature otherwise). The values to use must be the compound Accuracy value (uncertainty fixed + variable factor). NOTE THAT A&I must use the amended position/depth values whenever Hor/Vert uncertainties are encoded, irrespective of the value of the displayUncertainties attribute. This attribute is to only affect the display component of the horizontal uncertainties, as detailed in bullet point 1 above.
DavidGrant-NIWC commented 2 years ago

Jonathan,

Thanks - the data does load, but it has some errors.

image

image

happy to help with this one - apologies we haven't been able to prepare much data in support of the efforts here but we're doing some of the S-101 test data and can prepare test datasets for specific, simple cells. We have a YAML processor which produces S-101 conformant data. Attached cell has a QualityOfBathymetricData feature which works in v1.9.0 of the NIWC viewer. This is generated from the following YAML. If you want this modifying, let me know and I can produce a new version with the required attributes.....


  - Name: QualityOfBathymetricData
    Foid: 1810:1411:100
    Prim: Surface
    Attributes:
      - Name: categoryOfTemporalVariation
        Value: Unassessed
      - Name: dataAssessment
        Value: Unassessed
      - Name: featuresDetected
        id: 1
      - Name: significantFeaturesDetected
        parent: 1
        Value: false
      - Name: leastDepthOfDetectedFeaturesMeasured
        parent: 1
        Value: false
      - Name: surveyDateRange
        id: 2
      - Name: dateEnd
        parent: 2
        Value: 2021-01-01
      - Name: zoneOfConfidence
kusala9 commented 2 years ago

ah, yes of course.

remade file here...

101USTESTOR001.zip

try this one? I think this is valid to the latest FC (also now includes vertical uncertainty). If you want to change the values just let me know.

yaml should have been

  - Name: QualityOfBathymetricData
    Foid: 1810:1411:100
    Prim: Surface
    Attributes:
      - Name: categoryOfTemporalVariation
        Value: Unassessed
      - Name: dataAssessment
        Value: Unassessed
      - Name: featuresDetected
        id: 1
      - Name: significantFeaturesDetected
        parent: 1
        Value: false
      - Name: leastDepthOfDetectedFeaturesMeasured
        parent: 1
        Value: false
      - Name: surveyDateRange
        id: 2
      - Name: dateEnd
        Value: 20210101
        parent: 2
      - Name: zoneOfConfidence
        id: 3
      - Name: categoryOfZoneOfConfidenceInData
        Value: Zone of Confidence U
        parent: 3
      - Name: fullSeafloorCoverageAchieved
        Value: false
      - Name: verticalUncertainty
        id: 4
      - Name: uncertaintyFixed
        parent: 4
        Value: 0.5
kusala9 commented 2 years ago

this is the feature in JSON (easier to read).

    {
      "type": "Feature",
      "id": "1810:1411:100",
      "properties": {
        "featureType": "QualityOfBathymetricData",
        "surveyDateRange": {
          "dateEnd": "20210101"
        },
        "categoryOfTemporalVariation": "6",
        "featuresDetected": {
          "significantFeaturesDetected": "false",
          "leastDepthOfDetectedFeaturesMeasured": "false"
        },
        "fullSeafloorCoverageAchieved": "false",
        "zoneOfConfidence": {
          "verticalUncertainty": {
            "uncertaintyFixed": "0.5"
          },
          "categoryOfZoneOfConfidenceInData": "6"
        },
        "dataAssessment": "3"
      }
    }
DavidGrant-NIWC commented 2 years ago

I think that's the first dataset I've ever seen that doesn't have any errors!

Could you create the following two datasets to support prototyping/testing this issue?

Dataset 1 - model uncertainties using QoBD and SpatialQuality

Add two of each of the following point features, one with and one without displayUncertainties=true:

For the features which have displayUncertainties=true, associate an instance of SpatialQuality to the point.

Dataset 2 - model uncertainties using only QoBD

As above, but:

Note: Adding an aggregation between each QoBD and the features to which it pertains would eliminate the need for spatial evaluation in this encoding method.

kusala9 commented 2 years ago

yes, can do. I need to confirm some of the details with you but I'll do that via email/offline and we can work a test cell up.

DavidGrant-NIWC commented 2 years ago

Some results and questions arising from implementing portrayal and alerting on horizontalPositionUncertainty when associated via SpatialQuality: image

I'll start looking at implementing the circles for points with no associated SpatialQuality.spatialAccuracy, however:

DavidGrant-NIWC commented 2 years ago

Some additional results and questions.

I was able to implement for features which don't have an association to SpatialQuality, but:

Additional question:

alvarosanuy commented 2 years ago

@DavidGrant-NIWC - Can you please confirm the VG allocated to the magenta dashed line used to depict a feature's horizontalPositionUncertainty ? I think we agreed on a new dedicated VG to make it possible to turn the symbology On/Off by mariners as required.

DavidGrant-NIWC commented 2 years ago

@DavidGrant-NIWC - Can you please confirm the VG allocated to the magenta dashed line used to depict a feature's horizontalPositionUncertainty ? I think we agreed on a new dedicated VG to make it possible to turn the symbology On/Off by mariners as required.

alvarosanuy commented 1 year ago

Portrayal subWG meeting - 11th January 2023

  1. NIWC confirmed they haven't implemented vertical uncertainties; neither symbology nor A&I.

  2. Discussions continued regarding the need, or not, to display uncertainties (e.g dashed magenta circle). The alternative could be focusing the efforts on the development of A&I ('worst case scenario' option at mariner's request) during Route Planning only. No decision was made.

  3. This topic requires comprehensive mariners' input in order to identify user's needs, narrow down requirements and drive areas of development.

@DavidGrant-NIWC - Would the implementation of 'horizontal uncertainties' (portrayal and A&I) be available in PC 1.1.0 ????

Some discussion topics ... :

DavidGrant-NIWC commented 1 year ago

@DavidGrant-NIWC - Would the implementation of 'horizontal uncertainties' (portrayal and A&I) be available in PC 1.1.0 ????

No, there are too many issues to be resolved. In particular, if the current modeling is retained then S-100 Part 13 needs to updated; however, the proposal we presented at the S-100 WG to make this change was not accepted.

We can continue to prototype in a PC branch though.

DavidGrant-NIWC commented 1 year ago

No, there are too many issues to be resolved. In particular, if the current modeling is retained then S-100 Part 13 needs to updated; however, the proposal we presented at the S-100 WG to make this change was not accepted.

Proposal was rejected at TSM. Recommendation was to address by modifying the encoding/modeling: image

One of the following options would address the issue, and there are probably other ways I haven't considered:

Any changes to the modeling should probably also include DepthNoBottomFound.

alvarosanuy commented 1 year ago

Decisions made at Portrayal subWG meeting on 10/5/23

PC 1.2.0

  1. The first priority is on resolving 'spatial evaluation' issues.
  1. The second priority is on implementing A&I during Route Planning and Route Monitoring - Mariner has to be able to check a route taking into consideration the quality aspects (horizontal and vertical accuracies) of relevant hydrographic features (i.e. Sounding, UnderwaterRock, etc) lying within or close to a selected route. Reference: MSC.530.106.-.PERFORMANCE.STANDARDS.FOR.ELECTRONIC.CHART.DISPLAY.AND.INFORMATION.SYSTEMS.ECDIS.Nov2022 - sections 11.4.9 & 11.3.6: 'It should be possible for the mariner to select that the indications take into account accuracy information of relevant hydrographic information, as defined by IHO standards.'_ This performance would require a new S-164 test case.

PC 2.0.0

  1. Resolve the need for a dedicated symbology that portrays horiz & vert. accuracies of hydrographic features at mariner's request. In short, the need for the attribute 'display uncertainties' (mariner supported Use Case + decision around DF-ECDIS implementation).
mikan66 commented 1 year ago

Regarding point 1 above: "NIWC to submit paper to S101PT10 (June 2023) ....." noting S-101PT10 paper: "S-101PT10-07.9 29 May Vertical Datum Information in S-101", which references: https://iho.int/uploads/user/Services%20and%20Standards/S-100WG/S-101PT9/S-101PT9_2022_INF1-01_EN_Metadata_Issues_V1.pdf

This topic, point 1, falls into this Metadata issues paper as well. I plan to add talking points to Tom's presentation as needed during discussions at PT10.

TomRichardson6 commented 1 year ago

@mikan66 thanks Mikan no charge! you have enough actions on your plate! I would be happy for NIWC to provide its presentation from PT9 first when this arises on the agenda and I can then follow with my slides which really just seek to push towards a recommendation.

alvarosanuy commented 1 year ago

Decisions made at Portrayal subWG meeting on 18/10/23