tdwg / bdq

Biodiversity Data Quality (BDQ) Interest Group
https://github.com/tdwg/bdq
43 stars 7 forks source link

TG2-AMENDMENT_COUNTRYCODE_FROM_COORDINATES #73

Open iDigBioBot opened 6 years ago

iDigBioBot commented 6 years ago
TestField Value
GUID 8c5fe9c9-4ba9-49ef-b15a-9ccd0424e6ae
Label AMENDMENT_COUNTRYCODE_FROM_COORDINATES
Description Proposes an amendment to the value of dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary from the bdq:countryShapes that is attributable to a single valid country code.
TestType Amendment
Darwin Core Class dcterms:Location
Information Elements ActedUpon dwc:countryCode
Information Elements Consulted dwc:decimalLatitude
dwc:decimalLongitude
Expected Response EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude is bdq:Empty, or if dwc:countryCode is bdq:NotEmpty; FILLED_IN dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary in the bdq:sourceAuthority that is attributable to a single valid country code; otherwise NOT_AMENDED.
Data Quality Dimension Completeness
Term-Actions COUNTRYCODE_FROM_COORDINATES
Parameter(s) bdq:sourceAuthority
Source Authority bdq:sourceAuthority default = "10m-admin-1 boundaries UNION with Exclusive Economic Zones" {[https://www.naturalearthdata.com/downloads/10m-cultural-vectors/10m-admin-1-states-provinces/] spatial UNION [https://www.marineregions.org/downloads.php#marbound]}
Specification Last Updated 2024-08-18
Examples [dwc:decimalLatitude="-25.23", dwc:decimalLongitude="135.43", dwc:countryCode="": Response.status=FILLED_IN, Response.result=dwc:countryCode="AU", Response.comment="dwc:decimalLatitude and dwc:decimalLongitude contain interpretable values"]
[dwc:decimalLatitude="-38.280937", dwc:decimalLongitude="72.047790", dwc:countryCode="": Response.status=NOT_AMENDED, Response.result="", Response.comment="Coordinates do not fall in the boundary of any country"]
Source ALA, GBIF, iDigBio
References
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes This amendment simply fills dwc:countryCode from a lookup of dwc:decimalLatitude and dwc:decimalLongitude. dwc:coordinateUncertaintyInMeters and dwc:coordinatePrecicision (if present) imply a buffer around the provided coordinates. Likewise, country polygons cannot be 100% accurate at all scales (Dooley 2005), so a spatial buffer of the country boundaries is also justified. Taking spatial buffers into account does however greatly complicate the logic and the implementation of this and related tests. In this test, a detection of multiple country codes by sampling within the buffer while possible, is not considered.
ArthurChapman commented 1 year ago

Updated Expected Response, bdq:sourceAuthority and Description to replace bdq:sourceAuthority:[countryShapes] with bdq:countryShapes and Specification Last Updated

Changed Parameter(s) from bdq:sourceAuthority to bdq:countryShapes

Tasilee commented 1 year ago

I thought @tucotuco's suggestion was bdq:sourceAuthority_countryshapes (or maybe "countryShapes") etc? That way, we don't loose the source authority link.

ArthurChapman commented 1 year ago

We don't use that for any other bdq: terms But if all agree would could change it - but see the Vocab or #205 for all the terms we would have to change

| bdq:annotationAlertIf | [https://github.com/tdwg/bdq/issues/152] | | | bdq:countryShapes | [https://github.com/tdwg/bdq/issues/152] | | | bdq:annotationAnnotationSystem | [https://github.com/tdwg/bdq/issues/152] | | | bdq:defaultGeodeticDatum | [https://github.com/tdwg/bdq/issues/152] | | | bdq:earliestValidDate | [https://github.com/tdwg/bdq/issues/152] | | | bdq:geospatialLand | [https://github.com/tdwg/bdq/issues/152] | | | bdq:includeEventDate | [https://github.com/tdwg/bdq/issues/152] | | | bdq:latestValidDate | [https://github.com/tdwg/bdq/issues/152] | | | bdq:maximumValidDepthInMeters | [https://github.com/tdwg/bdq/issues/152] | | | bdq:maximumValidElevationInMeters | [https://github.com/tdwg/bdq/issues/152] | | | bdq:minimumValidDepthInMeters | [https://github.com/tdwg/bdq/issues/152] | | | bdq:minimumValidElevationInMeters | [https://github.com/tdwg/bdq/issues/152] | | | bdq:sourceAuthority | [https://github.com/tdwg/bdq/issues/152] | | | bdq:spatialBufferInMeters | [https://github.com/tdwg/bdq/issues/152] | | | bdq:targetCRS | [https://github.com/tdwg/bdq/issues/152] | | | bdq:taxonomyIsMarine | [https://github.com/tdwg/bdq/issues/152] | |

Tasilee commented 1 year ago

Thanks @ArthurChapman. The list includes terms that do relate to a bdq:sourceAuthority (annotations, country shapes/land, geodetic datum) but the rest don't really, unless you want to stretch the concept.

I have therefore changed my mind (backflip?) and think that the syntax like "bdq:countryShapes" seems expedient as all bdqs are listed under the 'heading' "Source Authority" anyway.

Tasilee commented 1 year ago

Aligned the Source Authority entries between #50 and this test

Tasilee commented 1 year ago

Post Zoom 11/7/2023, I have aligned the Source Authority with the suggested syntax:

bdq:sourceAuthority default = "ADM1 boundaries" {[https://gadm.org] spatial UNION with "Exclusive Economic Zones" [https://marineregions.org]}

Tasilee commented 1 year ago

In the light of #50 suggestion, changed Source Authority to

| Source Authority | bdq:sourceAuthority default = "ADM1 boundaries spatial UNION with Exclusive Economic Zones" {[https://gadm.org] spatial UNION [https://marineregions.org]} |

Tasilee commented 1 year ago

Changed Parameter(s) from bdq:countryShapes to bdq:sourceAuthority and presume we will leave it Parameterized.

Tasilee commented 1 year ago

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted".

Also changed "Field" to "TestField", "Output Type" to "TestType" and updated "Specification Last Updated"

Tasilee commented 6 months ago

Standardized reference to bdq:sourceAuthority in Expected Response to "EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available"

Tasilee commented 2 months ago

Changed Expected Response from

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude is EMPTY or uninterpretable, or if dwc:countryCode is NOT_EMPTY; FILLED_IN dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary from the bdq:countryShapes that is attributable to a single valid country code; otherwise NOT_AMENDED.

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude is EMPTY, or if dwc:countryCode is NOT_EMPTY; FILLED_IN dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary from the bdq:countryShapes that is attributable to a single valid country code; otherwise NOT_AMENDED.

Tasilee commented 1 month ago

The current Expected Response/Specification doesn't make sense as it uses bdq:countryShapes but this term is not defined, yet bdq:sourceAuthority is (and it already in the first part (EXTERNAL_PREREQUISITES....) that seems crazy given all the comments on this in this issue. Given the inconsistencies here could I suggest we change

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude is bdq:Empty, or if dwc:countryCode is bdq:NotEmpty; FILLED_IN dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary from the bdq:countryShapes that is attributable to a single valid country code; otherwise NOT_AMENDED.

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude is bdq:Empty, or if dwc:countryCode is bdq:NotEmpty; FILLED_IN dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary in the bdq:sourceAuthority that is attributable to a single valid country code; otherwise NOT_AMENDED.

??

chicoreus commented 1 month ago

Our current practice is to only give distinct names to sourceAuthority parameters when there is more than one in an issue. Thus, yes, it makes sense to change bdq:countryShapes here to the generic term bdq:sourceAuthority.

Tasilee commented 1 month ago

Done

ArthurChapman commented 1 month ago

As with the discussion under #98 and the issue of "High Seas" with dwc:countryCode-'XZ' See also #62.

If thought appropriate, we could change this test to something like

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude is bdq:Empty, or if dwc:countryCode is bdq:NotEmpty; FILLED_IN if 1) dwc:countryCode if dwc:decimalLatitude and dwc:decimalLongitude fall within a boundary in the bdq:sourceAuthority that is attributable to a single valid country code or 2) dwc:countryCode with 'XZ' if the coordinates fall with the boundary of "High Seas" as defined in [xxxxx]; otherwise NOT_AMENDED.

Or we can leave as it is now and just have "NOT_AMENDED. Just a suggestion.

Tasilee commented 1 month ago

As this test is an AMENDMENT, we are not implying lack of quality by a lack of an amendment. However, given the discussion, we could be proactive if we could set dwc:countryCode="XZ" using coordinates and a 'high seas' reference map.

@ArthurChapman: Your Expected Response is appealing. How say others?

tucotuco commented 1 month ago

We would also have to change the description if we change the expected response. I'm not fond of including the exception and complicating the test. Multipe bdq:sourceAuthorities would be required, and could each of those be parametrized?

I would not change this test. Rather, I would create set of tests around something like TG2-AMENDMENT-COUNTRYCODE_FROM_HIGH_SEAS_COORDINATES.

Tasilee commented 13 hours ago

I have changed dwc:decimalLatitude and dwc:decimalLongitude from Information Elements ActedUpon to Consulted. OK? This came from checking AMENDMENT relationships.

chicoreus commented 13 hours ago

@Tasilee good catch.