tdwg / bdq

Biodiversity Data Quality (BDQ) Interest Group
https://github.com/tdwg/bdq
43 stars 7 forks source link

TG2-AMENDMENT_ESTABLISHMENTMEANS_STANDARDIZED #269

Open ArthurChapman opened 6 months ago

ArthurChapman commented 6 months ago
TestField Value
GUID 15d15927-7a22-43f8-88d6-298f5eb45c4c
Label AMENDMENT_ESTABLISHMENTMEANS_STANDARDIZED
Description Propose amendment to the value of dwc:establishmentMeans using bdq:sourceAuthority.
TestType Amendment
Darwin Core Class dwc:Occurrence
Information Elements ActedUpon dwc:establishmentMeans
Information Elements Consulted
Expected Response EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL PREREQUISITES_NOT_MET if dwc:establishmentMeans is EMPTY; AMENDED the value of dwc:establishmentMeans if it can be unambiguously matched to a term in bdq:sourceAuthority; otherwise NOT_AMENDED
Data Quality Dimension Conformance
Term-Actions ESTABLISHMENTMEANS_STANDARDIZED
Parameter(s) bdq:sourceAuthority
Source Authority bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}
Specification Last Updated 2024-02-08
Examples [dwc:establishmentMeans="nat": Response.status=AMENDED, Response.result=dwc:establishmentMeans="native", Response.comment="dwc:establishmentMeans contains an interpretable value according to the bdq:sourceAuthority"]
[dwc:establishmentMeans="species": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:establishmentMeans does not contain an interpretable value according to the bdq:sourceAuthority"]
Source TG2
References
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes
tucotuco commented 6 months ago

The Notes are in error. "Unknown" is not a valid value for the vocabulary. The corresponding standard controlled value "uncertain " and the preferred label "uncertain (unknown, cryptogenic)". See https://dwc.tdwg.org/em/.

ArthurChapman commented 6 months ago

Like #268, maybe this should also be regarded as CORE now that there is a suitable Vocabulary. Again it received a rating of 6.5.

ArthurChapman commented 6 months ago

Thanks @tucotuco - correct. I was misreading and going by memory. I've deleted the Note.

chicoreus commented 6 months ago

@ArthurChapman I would concur, it makes good sense to include in core, some of the data quality needs elicited by TG3 do involve invasive species and the movement of species, so with vocabularies these tests do feel in scope for core.

Tasilee commented 6 months ago

I've removed the Note as we seem to agree this is now CORE.

ArthurChapman commented 6 months ago

The four tests (#277, #278, #268, #269) should be CORE (I have discussed this with Lee). Some reasons are

chicoreus commented 6 months ago

We should be using the following value for the source authority:

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/]}

See: https://github.com/tdwg/bdq/issues/268#issuecomment-1961631041

Tasilee commented 6 months ago

Thanks @chicoreus - Changing Source Authority from

[bdq:sourceAuthority default = "Darwin Core establishmentMeans" {[https://dwc.tdwg.org/list/#dwc_establishmentMeans]} {dwc:establishmentMeans vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

to

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/]}

tucotuco commented 5 months ago

I think https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts is OK as a source authority because it actually has an API, as long as it is understood that the actually vocabulary is maintained at https://dwc.tdwg.org/em/ and the GBIF API is expected to remain up to date with that.

Tasilee commented 5 months ago

https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts added back in. Dunno why it was changed back.

Tasilee commented 4 months ago

Changed Source Authority from

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

to

bdq:sourceAuthority default = "GBIF EstablishmentMeans Vocabulary" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans]} {"dwc:establishmentMeans vocabulary API" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

tucotuco commented 4 months ago

Source Authority should be

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

Tasilee commented 4 months ago

Changed Source Authority from

bdq:sourceAuthority default = "GBIF EstablishmentMeans Vocabulary" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans]} {"dwc:establishmentMeans vocabulary API" [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

to

bdq:sourceAuthority default = "Establishment Means Controlled Vocabulary List of Terms" {[https://dwc.tdwg.org/em/]} {GBIF vocabulary API [https://api.gbif.org/v1/vocabularies/EstablishmentMeans/concepts]}

chicoreus commented 4 months ago

The first example should be corrected to match the controlled vocabulary, and a disamiguation case that can be resolved from the controlled vocabulary, thus "Native" instead of "native" and "Native (Indigenous)" instead of "indigenous":

[dwc:establishmentMeans="Native (Indigenous)": Response.status=AMENDED, Response.result=dwc:establishmentMeans="Native", Response.comment="dwc:establishmentMeans contains an interpretable value according to the bdq:sourceAuthority"]

chicoreus commented 4 months ago

Looking harder, this is actually a problem with the GBIF controlled vocabulary, it differs from the the TDWG vocabulary in case. The correct value is "native", not "Native". Implementors will have to be aware that the expected value needs to be an exact match to the TDWG vocabulary. This applies to several tests.

[dwc:establishmentMeans="Native (Indigenous)": Response.status=AMENDED, Response.result=dwc:establishmentMeans="native", Response.comment="dwc:establishmentMeans contains an interpretable value according to the bdq:sourceAuthority"]

tucotuco commented 4 months ago

Since a ratified controlled vocabulary actually exists, we should urge GBIF to amend their implementation of the vocabulary.

chicoreus commented 4 months ago

See https://github.com/tdwg/bdq/issues/275#issuecomment-2061845648

The GBIF API is only of partial help here, it does not provide the actual Controlled Values from the TDWG vocabulary, the values it has differ in case, implementations can use the GBIF API for its additional values, but need to align the case to match the actual Controlled Value from the TDWG vocabulary for the dwc: term values (GBIF API is sufficient for the dwciri term values).

chicoreus commented 4 months ago

@tucotuco I agree.

Tasilee commented 3 months ago

GBIF vocabulary has now been aligned with Darwin Core. Thanks @timrobertson100