Input on validation checks

[rmalyankar]

The S-100 validation checks sub-group has before it a draft set of validation checks which affect S-102 (amongst other products). These checks will be discussed at a later meeting of the sub-group. I would like to draw the attention of the S-102 project team to the following checks from that list, so the discussion in the validation checks subgroup can take S-102 project team input, if any, into consideration.

Please note, this is not a comprehensive set of validation checks for S-102.

No.	Description	Inputs	Products	Classification
1	Horizontal Datum must be WGS84. horizontalDatumValue = 4326 and horizontalDatumReference = “EPSG”	Dataset	All	C
2	Depth must be negative up	Dataset	S-102	C
3	All depth values must be Real numbers	Dataset	S-102	C
4	"-90.0°<gridOriginLatitude<90.0° and -180.0°<gridOrginLongitude<180°"	Dataset	All	C
5	Dataset coverage must not overlap (temporal and spatial) >1	Dataset , CATALOG.XML, S-128 (service)	All	C
6	Well defined grid	Dataset	All	C
7	All grid dimensions > 0	Dataset	All	C
8	Grid Completeness. All grid points must have a valid value or noData value	Dataset	All	C
9	Grid Consistency. Correct number of values per row/column. Number of values in data must equal numPointsLongitudinal and numPointsLatitudinal from dataset metadata.			C
10	"Calculated grid from gridSpacing, numRows and numCols must not be outside defined coverage in metadata. Longitude Limit = GridOriginLongitude + (NumCol-1) * (gridSpacingLongitudinal). [from S-111 Eqn. 4.1] Latitude Limit = GridOriginLatitude + (NumRow-1)(gridSpacingLatitudinal). [from S-111 Eqn 4.2]"	Dataset	All	C
11	Regular Grid only is allowed for S-100 ECDIS (currently). S100_GridCoverage type (b) (Table 4.1 S-104). This is dataCodingFormat=2 in the dataset metadata. For S-102 this should be DCF 9	Dataset	S-102, S-104	C
12	interpolationType shall always be nearestNeighbour (see notes) _(notes: The method of interpolation used by S-100 ECDIS is also fixed as nearest neighbour. Although datasets may well use other interpolation types (as defined by S100_CVInterpolationMethod) the S-100 ECDIS will always use a nearest neighbour interpolation)	Dataset	S-102/ S-104	W
13	Maximum Resolution (See notes) See Note 2 below	Dataset vs Dataset	(S-102/S-104 ) and S-101	C
14	Minimum Resolution (See notes) See Note 2 below	Dataset vs Dataset	(S-102/S-104) and S-101	C
15	Vertical Datum/Sounding Datum must be the same in areas of overlapping coverage or WLA/USSC can not be computed. WLA=Water Level Adjustment, USSC=User Selected Safety Contour	Dataset vs Dataset	S-101, S-102, S-104	C
16	"S-102 depth values (excluding noData, defined by bounding rectangle) shall only spatially intersect the following Group 1 features (S-101): 1. Depth Area 2. Dredged Area"	Dataset vs Dataset	S-102/S-101	C
17	"S-102 values within defining bounding box should not be shoaler than 1. DepthArea.DepthRangeMinimumValue (any feature as specified by feature catalogue) 2. Sounding depth value (ZCOO/COMFZ) 3. ValueOfSounding (any feature with valueOfSounding in S-101 feature catalogue)"	Dataset vs Dataset	S-102/S-101	C
18	Where polygons with depth attribution (S-101) are not enclosed by S-102 coverage, user selected safety contour can not be calculated.	Dataset vs Dataset	S-101 / S-102	W

All in the Products column means all data products. C/E/W in the Clasification column is the severity of the check failure: Critical, Error, Warning

Notes:

1. As mentioned earlier, this is not a comprehensive set of validation checks for S-102 datasets, just "S-100 level" validation checks. The comprehensive list of dataset validation checks is being prepared separately and will be circulated later.
2. The issue of whether a minimum or maximum resolution should be imposed via validation should be discussed with the S-100WG as a whole as it has a bearing on S-100 ECDIS implementers, as well as data producers. In terms of minimum resolution it is likely that unless data is of a certain minimum threshold resolution (in relation to the largest scale ENC coverage corresponding to the dataset, then both User Selected Safety Contour and Water Level Adjustment will be unusable (and, in all cases, worse than the corresponding ENC). Although the user is free to not use S-102/S-104 in these cases it would seem appropriate to put a minimum resolution in place to ensure data is fit for purpose, given the extensive use likely to be made of this feature. Similarly, a maximum resolution is probably appropriate, in order to provide ECDIS implementers with a realistic estimate of maximum data volumes/density expected. A measure of maximum density may be found either in the average pixel size onscreen or via the old 0.3mm rule (which determines the minimum distance between vertices under S-57 ENC). Assuming the same 0.3mm, as measured at MaximumDisplayScale a maximum resolution could be recommended. The logic of this is that at higher resolution this requires the OEM to resolve the display of multiple depth values with the same (displayed) physical location to the user, as well as their integration into the user selected safety contour and water level adjustment sections of S-98 Annex C. A consultation with data producers and OEMs is probably the best way to define any validation tests over minimum/maximum resolution for S-102 (and possibly S-104) within S-100 ECDIS and the results used to refine the validation tests noted in this report.
3. Some tests, e.g., items 6 ("well defined grid") or 3 ("depth values must be real numbers"), may be unclear, or unnecessary as a validation check (the latter by virtue of being precluded by the specific HDF5 data type or other format constraint). Feel free to point out unclear, ambiguous, or unnecessary checks where appropriate.

[RohdeBSH]

Hello @rmalyankar,

Thank you for pointing out the developments of Validation Checks. These are my initial thoughts from my vacation, so please don't take them personally. I just wanted to note these already.

Item 1: This is an absolute no-go. The S-102 in chapter 5.2 table 1 describes all valid coordinate systems for the S-102. There is not only EPSG:4326 contained, but for example also EPSG:32632. I cannot support the omission of projected coordinate systems on the basis of WGS84. At the same time, the manual definition of coordinate system according to S-100 Ed. 5.0.0 becomes senseless. I mean the points 5-19 in the table 8 in the chapter 10.2.1 of the S-102.

Item 2: How is this going to be tested? I am thinking of tide-dependent areas such as the North Sea. Data sets can contain positive and negative values. How should it be checked whether the data set adheres to "Depth must be negative up"?

Item 4: This is not possible either, or only if point 1 prevails. Because in the S-102 this is either easting/northing or lat/lon depending on the chosen coordinate system (S-102 chapter 10.2.5 table 12). The projected coordinate systems are outside the range of values.

Item 13/14: I do not understand the note 2. When we discuss this, a graphical elaboration would be good.

Item 16: What about LandArea with the geometry type point? Should we remove the grid cell there? As with us, create a 1x1 meter hole at best and a 10x10 meter hole at worst?

Item 17: I think that's nonsense. The S-102 can be produced much faster than the S-101, at least for us. We need at most one day to update the S-102, but up to a week (if not longer) for the S-101. So it goes without saying that the S-102 can be shallower at times. This rule would mean that S-102 and S-101 would always have to be synchronous. In my opinion, the S-102 should just quickly deliver more current data, because we all know that the S-101 production can take longer. From my point of view, the rule makes the S-102 almost superfluous for navigation.

Item 18: I think that's nonsense, too. The S-101 can be much larger in extent than the S-102. In our case, for example, the S-101 ENC is a multiple of the S-102. This also means that possibly an S-101 ENC is not 100% covered by S-102 products. For larger depth areas that only have partial S-102 coverage, would the S-102 then not be displayed? I don't see the logic of this. Sure, depth contours created from the S-102 would suddenly just end in the middle of the depth area. But where is the problem? Who does not want that should switch off the S-102. Have you asked the mariners and pilots about this?

As I said, these are just my quick thoughts. Please do not take it personally. ;)

[rmalyankar]

That's alll right. Continue to elaborate on these comments. Comments from other members of the S-102 project team, either supporting or disagreeing, are invited.

[rmalyankar]

Item 2: How is this going to be tested? I am thinking of tide-dependent areas such as the North Sea. Data sets can contain positive and negative values. How should it be checked whether the data set adheres to "Depth must be negative up"?

Perhaps by restricting verticalCS to 6498 (in Table 8 in ed. 2.2) combined with producer verification (and attestation?) that their production tools actually do generate data confoming to the positive-down rule? Further verified by human spot-checking of random datasets?

[PalmBSH]

Perhaps by restricting verticalCS to 6498 (in Table 8 in ed. 2.2) combined with producer verification (and attestation?) that their production tools actually do generate data confoming to the positive-down rule? Further verified by human spot-checking of random datasets?

I agree with you to restrict verticalCS to 6598 but this is not a guarantee for the correct direction in the software. We see this with our data suppliers, they provide the depths referenced to a projected coordinate system with positive axis orientation up. And it does not help to validate the depth values.

[tfilppula]

@RohdeBSH, @rmalyankar about the validation test 1: Instead of testing specifically for horizontal datum I propose to test for EPSG value in horizontalCRS root group attribute. This value must be one of the allowed values defined in PS section 5.2 Table 1:

In my opinion this should be sufficient as all of the listed CRSs are based on WGS84 datum.

[RohdeBSH]

That sounds good and correct from my point of view. However, as it is described in item 1, only EPSG:4326 would be permitted and that is not acceptable from my point of view.

Also, the attributes "horizontalDatumValue" and "horizontalDatumReference" still refer to the root group of edition 2.1.0 and not to 2.2.0, so we have to be careful not to mix it up. In the edition 2.2.0 only the attribute "horizontalCRS" has to be checked, as @tfilppula mentioned.

[poseiron01]

Hi all, Good discussions! I would also like to comment No 5. Does this check somehow also include products coverage between countries or just in your own portfolio? Otherwise I imagine that in the border regions it’s bound to be some overlap. In S-98 (Annex C edition 1.0.0) section C-4-1.2 it states: “S-102 data is not expected to overlap, but in case of overlap the ECDIS should indicate an overlap by the text “OVERLAP” and the user should have the ability to select which producer in the overlapped area has priority and will be selected for processing safety contour”. This seem to be in conflict with this check if products from different producers are included. Please correct me if I'm wrong.

[poseiron01]

On No 17 we agree with BSH. Our future production of S-102 will also be much faster than for S-101 which might take weeks in some cases. We have discussed at SMA developing some kind of check between the new S-102 and the published ENC (not yet updated with the new bathymetry) to see if there was a critical conflict before releasing the S-102, but have not landed in anything yet. Anyhow any major hazards to navigation are taken care of either during the survey itself with a report from the ship or in the following QA-process at the office. Normally a Notice to Mariners goes out and the ENC is updated promptly. So if there is any conflict at this stage it should only be a minor one in for example deeper areas, not relevant for the ships. We feel that more current data should always go out to the mariners as soon and fast as possible.

[rmalyankar]

Good comments all, please continue to comment as issues and cases occur to you.

On No. 5, in addition to @poseiron01's comment about different producers, is there a use case for a single producer publishing overlapping (or even coincident) S-102 datasets at different resolutions? (Given that S-102 does not yet provide for multi-resolution datasets - Annex E is just a stub at present.)

I think No. 16 also has the same issue as No. 17, namely "which product is more up-to-date"? The part about "defined by bounding rectangle" also needs to be clarified, the S-101 features will probably use smoothed curves.

[rmalyankar]

My own comments.

No.	Comment
1	Doesn't work for UTM as others have written, but CRS should be consistent with underlying ENC
2	Agree. S-102 revision needed (see previous comments)
3, 8	Superfluous, data type and range check on data values would catch these
4	Should be rewritten in terms of the CRS actually used, see validation checks annex spreadsheet
5	TBD
6	I don't understand this check
7	OK
9	OK
10	Implemented in a different form in validation annex spreadsheet, in terms of instance bounding box.
11	OK for S-102, S-104 is for TWCWG to comment on
12	OK iff bathymetric coverage model uses this. WLA can use nearestNeighbour as an approximation for its own purposes regardless
13, 14	Presumably still to be discussed in some forum - where?
15	"overlapping" with S-101, presumably?
16, 17, 18	I disagree with these checks as written and concur with the arguments objecting to them in comments above

[kusala9]

As the author of the original tests :-) it's good to see the discussion so far, I wonder if it's worth some of them being broken out into separate issues? Certainly at the validation meeting in Lombok it would be good to go through some of the details. As I've not had the chance to present the thinking behind them, by way of comment a bit of background might be useful....

• These were very much meant as "draft" tests, and ideas for them so refinements to the way we express them would (and are) much appreciated.
• They are purely focused on the ECDIS use case, not on any broader uses of S-102, and so should only be read in the context of ECDIS use of S-102.
• 16,17,18 - these tests are NOT S-102 validation tests, they're tests which are meant to express any vertical inconsistency between S-102 and the (I guess) largest scale ENC which intersects with the S-102. I still believe that an S-102 which contains data around which a safety contour would be drawn (under S-98 Annex C) whereas the ENC would NOT contain a feature which would cause a similar indication on the ECDIS constitutes a vertical inconsistency between the S-102 and the S-101. Maybe this should be a warning rather than a critical error but I believe the data producer will want to know if such things exist, and take any mitigating actions. That justifies (in my mind) the existence of a validation check for such conditions. that's my justification for 17. So, maybe turn it into a warning, then data producers can mitigate the risk by NM or NW?? either way, it's an inconsistency. this is one that must be looked at from the end user/ECDIS perspective. As stated this is not an "invalid" S-102, it's an S-102 which is inconsistent (possibly, justifiably so) with the largest scale ENC.
• 16, should there be S-102 where the largest scale ENC has "dry" group 1 features? In the same way as the previous point, if I'm an end user, this isn't great. S-102 is not a "larger scale chart" - it's an overlay onto the ENC in terms of the ECDIS so harmonised portrayal will be adversely affected if the Land Areas are overlaid with depths. Again though, maybe this is a warning, not a critical error but it needs consideration.
• 18 is a statement of fact on the way water level adjustment under S-98 Annex C works. If the conditions aren't met then WLA can not be calculated by the ECDIS and the user is informed of them. hence, it's a warning, not a critical error as it impacts the ability of the ECDIS to perform a user function. Again, it concerns the interaction of the ENC with the S-102 and the existence of the test highlights a potential inconsistency that the data producer should know about (and possibly mitigate) prior to distribution to end users.

The arguments about S-102 being more up to date than the S-101 are accepted. It may be that data producers can mitigate such inconsistencies to their own satisfaction, so maybe in many cases these can be accepted. That doesn't take the vertical inconsistency away though and data producers (those responsible for distribution and evaluation of any liability/mitigating actions) would want to run tests against data prior to its release. Hence, a validation test is required. Who runs it, when they run it and what they do with the results maybe needs to be answered in the context of multi-product distribution? Bear in mind that during the transition period many ECDIS will be S-57 ENC only and S-102 is highly unlikely to be mandated, and S-101 will discharge all SOLAS obligations. The user will believe if they have the ENC on board (with any navwarn, T&P, local warnings etc etc) then they are fully compliant. S-102 would therefore need to be supplemented (as many have said) with other messages mitigating such inconsistencies. A standardised validation test means IHO has an acknowledged way of establishing them rather than all data producers developing their own.

[rmalyankar]

They are written as validation checks, lots of them are ordinary validation checks, they're before the S-100 validation checks group...should they even be before that group?

[kusala9]

there's been a couple of discussion on this topic in the last week to add to this thread. In the S-164 group a breakout meeting on implementing water level adjustment with the OEMs specifically requested that IHO puts validation tests in place to ensure compatibility between S-101 and S-102/S-104 data. The requested tests were

• identical vertical (sounding) datums
• no holes in the S-102/S-104 coverage unless over land and no coverage on land
• use of only regular grids
• coincident coverage of S-102/S-104

from the OEM perspective, all S-102/S-104 data destined for ECDIS (and this is only for data for use on an ECDIS) should meet these requirements - the OEM will filter out anything not meeting them and tests have been requested for S-164 so they can develop that functionality.

The validation subgroup of the S-101PT will put a paper to S100WG proposing that tests between product specificaitons (i.e. compatibility tests) should be established at the S-100 level and then an appropriate place found to keep them (maybe another annex in S-98). The proposed tests above will be drafted into there - I suggest a round table at S100WG to see if we can come up with good wording for them from both the S-101 and TWCWG groups? S-164 group will separately propose that a clause is inserted into both S-102 and S-104 detailing the compatibility requirements of data with S-101, which is destined for ECDIS.

[kusala9]

this was the picture shown be 7Cs at the S-164 meeting. From their point of view they would not want to calculate WLA or user selected safety contour on the nodata holes in the S-102. It's ok if the holes in the data are over land but not if there's depth areas underneath them. It may be that we can rephrase this test in some way but I think the issue (from the OEMs perspective) is clear here.

[kusala9]

In discussions with IEC it seems highly unlikely that S-100 ECDIS will be able to deal with the implications of overlapping S-102/S-104, particularly in the context of WLA and user selected safety contour. Test 5 on overlapping coverages should be added to any evolving list of validation tests. There's a proposal to add clarifying words into S-98 Annex C on this point.

[RohdeBSH]

Hi @kusala9,

Holes

How is a hole defined? In respect to a grid cell, the neighborhood of the other grid cells can be 4 or 8. Is it a hole if a grid cell is empty and the neighbor cells are filled? Or is it also a hole if only three or seven neighbor cells are filled. This can be a problem at the borders of the product.

How big is a hole? Does it have to be exactly the grid resolution, or can it be a multiple of the grid resolution? What about islands? Sure, with an island there is a land area. But it is not always possible to survey to the coastline. For example, it may be too shallow, or as in Rostock, Germany simply a nature reserve that may not be navigated. Thus, there is an unsurveyed strip around the island. Is this strip a hole?

No coverage on land

As you can see in the picture, measurements have been taken under the quay. This is simply due to the construction of the quay. It's not a mistake, it's just unsightly. How should we now cut the quay from the grid data in such a way that there are no holes? That is exactly the problem between raster and vector data. Our idea would be, the LNDARE is generally displayed by the ECDIS above the S-102. Quasi, the depths measured under the quay are faded out or overlaid by land.

[RohdeBSH]

By the way. Isn't the data from your image still better than no data at all? Okay, the safety contour can't be drawn accurately, but mariner still gets much more information from the data than just the safety contour. He can get his own impression of the seafloor, which helps him make his decisions.

[gseroka]

In discussions with IEC it seems highly unlikely that S-100 ECDIS will be able to deal with the implications of overlapping S-102/S-104, particularly in the context of WLA and user selected safety contour. Test 5 on overlapping coverages should be added to any evolving list of validation tests. There's a proposal to add clarifying words into S-98 Annex C on this point.

@kusala9 what is meant by "overlapping S-102/S-104" in this case? ("Dataset coverage must not overlap (temporal and spatial) >1")?