search

tdwg / bdq

Biodiversity Data Quality (BDQ) Interest Group
https://github.com/tdwg/bdq
43 stars 7 forks source link

TG2-VALIDATION_GEODETICDATUM_STANDARD #59

Open iDigBioBot opened 6 years ago

iDigBioBot commented 6 years ago
TestField Value
GUID 7e0c0418-fe16-4a39-98bd-80e19d95b9d1
Label VALIDATION_GEODETICDATUM_STANDARD
Description Does the value of dwc:geodeticDatum occur as a valid geographic CRS, geodetic Datum or ellipsoid in bdq:sourceAuthority?
TestType Validation
Darwin Core Class dcterms:Location
Information Elements ActedUpon dwc:geodeticDatum
Information Elements Consulted
Expected Response EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; COMPLIANT if the value of dwc:geodeticDatum is (1) "not recorded" or (2) a valid geographic EPSG code for a CRS, Datum, or ellipsoid; otherwise NOT_COMPLIANT
Data Quality Dimension Conformance
Term-Actions GEODETICDATUM_STANDARD
Parameter(s)
Source Authority bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}
Specification Last Updated 2023-09-17
Examples [dwc:geodeticDatum="epsg:4326": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:geodeticDatum matches an unambiguous alphanumeric CRS or datum code value in the bdq:sourceAuthority"]
[dwc:geodeticDatum="7030": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:geodeticDatum doesn't match values in the bdq:sourceAuthority, 1730 (EPSG:1730) is an ellipsoid not a datum"]
Source ALA, GBIF
References
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes Darwin Core recommends best practice is to use a controlled vocabulary. This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. Chapman and Wieczorek (2020) recommend best practice is to use EPSG geographic CRS or Datum codes (https://epsg.io/) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326".
iDigBioBot commented 6 years ago

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet: 200 meters refers to only a few datums (Australia?) - In NAD27-WGS84 can be as high as 480 meters (from memory) in Aleutian Islands, and greatest distance is around 3,520 meters with an Indian Datum (again from memory)

iDigBioBot commented 6 years ago

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet: For NOTES Column

iDigBioBot commented 6 years ago

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet: Should we change name to GEODETIC_DATUM_AMBIGUOUS and make warning type Ambiguous?

godfoder commented 6 years ago

img_20180117_104010

chicoreus commented 5 years ago

Parameter is not needed for this test, the vocabulary expected by dwc;geodeticDatum is the EPSG vocabulary. Parameter might be needed to specify if the expected values are in the form https://epsg.io/4326, EPSG:4326, or WGS84, but the EPSG vocabulary is the one that everyone converges on, and different user communities are not likely to want different vocabularies for this test. The specification of the vocabulary should go into the specification or into the notes, not a parameter.

ArthurChapman commented 5 years ago

Agreed. Just accept https://epsg.io/ as the bdq:sourceAuthority and a value is valid if it is in that vocabulary. Perhaps we should format it as being a valid EPSG Code (as opposed to a Datum Code)

Tasilee commented 5 years ago

Agreed, but again, "https://epsg.io/" in References?

ArthurChapman commented 2 years ago

I have added "epsg:4326" to the examples, but shouldn't we delete WGS:84 and GD66 as valid examples when in #60 we suggest an amendment of WGS84 to epsg:4326?

tucotuco commented 2 years ago

This one is a tough one in that epsg isn't actually a standard either. But it is the closest we have and it is what Darwin Core recommends to use, if possible. Nevertheless, the EXPECTED_RESPONSE says, "or an unambiguous alphanumeric CRS or datum code", which makes all the examples in the Darwin Core definition valid, namely: EPSG:4326, WGS84, NAD27, Campo Inchauspe, European 1950, Clarke 1866, unknown. I think that all of those should trigger a COMPLIANT response. I would use all those examples and add, epsg:4326, and 4326.

Tasilee commented 2 years ago

I've added "4326"

Tasilee commented 2 years ago

Added to Notes: "This test will fail if there are leading or trailing white space or non-printing characters."

Tasilee commented 1 year ago

This is a 'test' where we reference bdq:sourceAuthority under INTERNAL_PREREQUISITES_NOT_MET but then use "is a valid EPSG CRS Code (with or without the "epsg" namespace prepended), or an unambiguous alphanumeric CRS or datum code;" subsequently. Should we reword it to something like

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; COMPLIANT if the value of dwc:geodeticDatum is a valid EPSG CRS Code (with or without the "epsg" namespace prepended) in bdq:sourceAuthority, or an unambiguous alphanumeric CRS or datum code in bdq:sourceAuthority; otherwise NOT_COMPLIANT

or is that overkill?

tucotuco commented 1 year ago

@Tasilee Reading the rewrite struck me as odd. Made me pause. It doesn't seem wrong on inspection, but I am not sure it adds to the clarity of the test.

ArthurChapman commented 1 year ago

Changed epsg: to EPSG: in several places

ArthurChapman commented 1 year ago

Should we add the more detailed Note under #60 to this test as well?

Tasilee commented 1 year ago

Agreed and done

ArthurChapman commented 1 year ago

On advice from EPSG - changed in References: "https://www.epsg.org/" to "https://epsg.org/"

Tasilee commented 1 year ago

Changed Expected Response (and Specification Last Updated) from

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; COMPLIANT if the value of dwc:geodeticDatum is a valid EPSG code; otherwise NOT_COMPLIANT

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; COMPLIANT if the value of dwc:geodeticDatum is a valid EPSG CRS or datum code; otherwise NOT_COMPLIANT

Tasilee commented 1 year ago

Changed Expected Response from

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; COMPLIANT if the value of dwc:geodeticDatum is a valid EPSG CRS or Datum code applicable to geographic coordinates in degrees; otherwise NOT_COMPLIANT

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available, INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; COMPLIANT if the value of dwc:geodeticDatum is a valid EPSG CRS or Datum code applicable to geographic coordinates in degrees; otherwise NOT_COMPLIANT

ArthurChapman commented 1 year ago

Following ZOOM discussion, updated Expected Response, Description and Specification Last Updated to restrict application of the test to Datums applicable to geographic coordinates etc.

Tasilee commented 12 months ago

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted". Also changed "Field" to "TestField" and "Output Type" to "TestType".

chicoreus commented 6 months ago

Updated comment from "fail" to more specific "This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. "

Tasilee commented 4 months ago

Changed Source Authority from

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}

to

bdq:sourceAuthority default = "GBIF GeodeticDatum Vocabulary" [https://api.gbif.org/v1/vocabularies/GeodeticDatum]} {"dwc:lifeStage vocabulary API" [https://api.gbif.org/v1/vocabularies/GeodeticDatum/concepts]}

ArthurChapman commented 4 months ago

@Tasilee - some errors in that - in the second part you mention Life Stage - I thought we were following EPSG and not GBIF for GeodeticDatum - or have I missed something @tucotuco ?

tucotuco commented 4 months ago

EPSG for dwc:geodeticDatum

Tasilee commented 4 months ago

Reverted to EPSG

chicoreus commented 4 months ago

Might be worth a note that

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} [https://api.gbif.org/v1/vocabularies/GeodeticDatum]} {"GBIF GeodeticDatum API" [https://api.gbif.org/v1/vocabularies/GeodeticDatum/concepts]}

May be a viable alternative sourceAuthority, as the GBIF GeodeticDatum API appears to have name values that match EPSG codes. Still, since the authority EPSG themselves provide and API, we should definitely use that in the default.

ArthurChapman commented 4 months ago

@chicoreus - I agree. EPSG will always be up to date as it is the authority - GBIF may try to keep up to date, but could lag.