tdwg / bdq

Biodiversity Data Quality (BDQ) Interest Group
https://github.com/tdwg/bdq
43 stars 7 forks source link

BDQ Core - VOCABULARY of terms #152

Open Tasilee opened 5 years ago

Tasilee commented 5 years ago

Terms in the bdqffdq namespace are from the Fitness for Use Framework (Viega et al. 2017). Use the reference to the Framework Definitions for more details and examples. The use of a vocabulary term in a test specification without a namespace prefix (sometimes represented in all UPPER CASE), implies that the bdq: or bdqffdq: namespace is applicable. Note that wherever "DQ" is used in a definition it implies "Data Quality" and wherever "FFU Framework" is used it refers to the "Fitness for Use Framework" (Veiga et al. 2017).

namespace:Term Term Definition Context Comment
bdqffdq:ActedUpon ActedUpon A bdqffdq:InformationElement that is the primary focus of a test bdqffdq:InformationElement
bdq:Alien-Species Alien-Species Research uses for occurrence data of alien species where 1) the information elements concern what organism occurred where and when and the means, degree, and pathways of establishment, and 2) that are used for analysis of spatial and/or temporal patterns of biodiversity (see examples in Groom et al. (2019). Improving Darwin Core for research and management of alien species. https://doi.org/10.3897/biss.3.38084). bdqffdq:UseCase
bdq:Ambiguous Ambiguous Used to report where bdq:Conformance is not satisfied due to bdqffdq:InformationElements not being unambiguously resolvable by a bdq:sourceAuthority. bdqffdq:DataQualityDimension
bdq:AMENDED AMENDED Used to indicate that a response for a bdqffdq:Amendment contains a proposed change to a record in the bdq:Response.result. bdq:Response.status Applies only to a bdqffdq:Amendment.
bdqffdq:Amendment Amendment A Data Quality needs level concept that describes a run of a test that proposes changes based on some data quality enhancement. The AMENDMENT concept in the Tests involves data that were amended by modification or addition of a value or values following defined bdqffdq:Criteria of a run result that includes a status of (bdq:AMENDED, bdq:FILLED_IN, bdq:TRANSPOSED, etc) as well as the proposed changes to values from the original data. FFU Framework: Class Formally in the Fitness for Use Framework (Veiga et al. 2017), the description of a test that can propose a change is a bdqffdq:Enhancement, while the corresponding report level concept is a bdqffdq:Amendment.
bdqffdq:AmendmentMethod AmendmentMethod A Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Enhancement in the context of bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements. FFU Framework: Class Veiga et al. (2017).
bdqffdq:AmendmentPolicy AmendmentPolicy A Data Quality needs level concept that describes how some bdqffdq:contextualizedEnhancement relates to a bdqffdq:UseCase. This relationship defines which amendments are supported by a given use case. FFU Framework: Class Veiga et al. (2017).
bdqffdq:amendmentProperties amendmentProperties Sub properties of bdqffdq:ObjectProperties that apply to amendment concepts such as bdqffdq:AmendmentPolicy (DQ needs), bdqffdq:AmendmentMethod (DQ solutions) and bdqffdq:Amendment (DQ reports). FFU Framework: ObjectProperty Veiga et al. (2017).
bdqffdq:AmendmentReport AmendmentReport A Data Quality report level concept that results from a bdqffdq:Amendment that proposed changes to a bdqffdq:InformationElement. FFU Framework: Class Veiga et al. (2017).
bdq:annotationAlertIf annotationAlertIf Optionally establishes if an annotation exists within a bdq:annotationSystem by describing the criteria for relating annotations in the system to records in a bdq:ParameterizedTest. bdq:Parameter Used in test "ANNOTATION_ISSUE_NOTEMPTY" (fecaa8a3-bbd8-4c5a-a424-13c37c4bb7b1).
bdq:annotationSystem annotationSystem Optionally established a system for annotations within a bdq:ParameterizedTest with the default being the w3c Annotations Data Model's "oa:Annotation" bdq:Parameter Used in test "ANNOTATION_ISSUE_NOTEMPTY" (fecaa8a3-bbd8-4c5a-a424-13c37c4bb7b1).
bdqffdq:Assertion Assertion The bdqffdq:Assertion type in FFDQ is the fundamental concept that makes up a bdqffdq:DataQualityReport. bdqffdq:Assertion can be any one of four types (represented as subClasses), bdqffdq:Measure, bdqffdq:Validation, bdqffdq:Issue, and bdqffdq:Amendement. FFU Framework: Class Veiga et al. (2017). The assertion concept consists of a bdqffdq:Specification (the technical description of a performed test), a bdqffdq:DataResource (initial values of input data expressed in terms of some controlled vocabulary), the bdqffdq:Mechanism (external service, actor, or code that performs the test), and some form of bdqffdq:Result.
bdq:ASSUMEDDEFAULT ASSUMEDDEFAULT A bdqffdq:Amendment that replaces a bdq:EMPTY term with a predefined default bdq:Parameter value. bdq:Response Would be used only in an extension or in bdq:Response.comment, bdq:Response.status value for this case is bdq:AMENDED.
bdq:Biotic-Relationships Biotic-Relationships Research uses for relationships between organisms where 1) the information elements concern what organisms have a relationship and 2) that are used for analysis of the relationship of one organism to another (see examples in ​​Poelen JH, Simons JD, Mungall CJ. (2014). Global Biotic Interactions: An open infrastructure to share and analyze species-interaction datasets. Ecological Informatics, 24, 148–159. https://doi.org/10.1016/j.ecoinf.2014.08.005) bdqffdq:UseCase
bdq:COMPLETE COMPLETE A bdqffdq:Assertion of a bdqffdq:Measure where data are present and sufficiently comprehensive for use. bdq:Response.result
bdq:Completeness Completeness The extent to which data are present and sufficiently comprehensive for use. bdqffdq:DataQualityDimension Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:Conformance Conformance Conforms to a format, syntax, data type, range, or standard of the bdqffdq:InformationElement. bdqffdq:DataQualityDimension Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:COMPLIANT COMPLIANT A bdq:ExpectedResponse of a bdqffdq:Validation where the data conforms to the test bdqffdq:Criterion. bdq:Response.result Applies only to bdqffdq:Validations.
bdqffdq:ComposedOf ComposedOf Describes the properties from a controlled vocabulary that compose a bdqffdq:InformationElement. For example, a bdqffdq:InformationElement may be bdqffdq:composedOf properties such as dwc:day, dwc:month and dwc:year. FFU Framework: ObjectProperty Veiga et al. (2017).
bdq:Consistency Consistency Agreement among related bdqffdq:InformationElements that are present in the data. Note that missing bdqffdq:InformationElements do not make a test bdq:Inconsistent. bdqffdq:DataQualityDimension Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:CONSISTENT CONSISTENT Identifies inconsistency among values between bdqffdq:InformationElements. bdq:Response.result
bdqffdq:Consulted Consulted A bdqffdq:InformationElement that was referenced in a test but was not the primary focus of the test. bdqffdq:InformationElements
bdqffdq:ContextualizedCriterion ContextualizedCriterion Describes an instance of the criterion concept in terms of the associated bdqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord. FFU Framework: Class Veiga et al. (2017).
bdqffdq:ContextualizedDimension ContextualizedDimension Describes an instance of the bdqffdq:Dimension concept in terms of the associated bndqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord. FFU Framework: Class Veiga et al. (2017).
bdqffdq:ContextualizedEnhancement ContextualizedEnhancement Describes an instance of the bdqffdq:Enhancement concept in terms of the associated bdqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord. FFU Framework: Class Veiga et al. (2017).
bdqffdq:ContextualizedIssue ContextualizedIssue Describes an instance of the bdqffdq:Issue concept in terms of the associated bdqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord. FFU Framework: Class Veiga et al. (2017).
bdq:CONVERTED CONVERTED A conversion has been proposed to values in the bdqffdq:InformationElements to conform with a targeted reference system. bdq:Response See Test "AMENDMENT_COORDINATES_CONVERTED" (620749b9-7d9c-4890-97d2-be3d1cde6da8).
bdq:COORDINATES COORDINATES Represents the combination of the Darwin Core terms dwc:decimalLatitude and dwc:decimalLongitude. bdqffdq:InformationElement
bdqffdq:coversUseCase coversUseCase Used by concepts in the Data Quality needs category to describe the relationship between DQ Policies (bdqffdq:ValidationPolicy, bdqffdq:AmendmentPolicy, bdqffdq:MeasurementPolicy) and an instance of the bdqffdq:UseCase covered by that policy. FFU Framework: ObjectProperty Veiga et al. (2017).
bdqffdq:Criterion Criterion Describes the criterion a bdqffdq:Validation test uses to determine compliance. For example, "The value of dwc:basisOfRecord of bdqffdq:SingleRecords must be in the controlled vocabulary". FFU Framework: Class Veiga et al (2017).
bdqffdq:criterionInContext criterionInContext Describes the relationship between a bdqffdq:Validation concept in the FFU Framework (needs, solutions, reports) and a bdqffdq:contextualizedCriterion. FFU Framework: ObjectProperty Veiga et al. (2017).
bdq:dataID dataID The local (to bdq:ValidationData) integer indentifier for the Validation Data record bdq:ValidationData
bdqffdq:DataQualityDimension DataQualityDimension Describes the aspect of data quality (accuracy, precision, completeness, etc.) that a test examines. For example, "precision" in "coordinate percision of single records". Includes Completeness (q.v.), Conformance (q.v.), Consistency (q.v.), Likeliness (q.v.), Reliability (q.v.), and Resolution (q.v.). FFU Framework: Class Note that the fail (bdq:NOT_COMPLIANT) assertions from running a test are one of: bdq:Ambiguous, bdq:Incomplete, bdq:Inconsistent, bdq:Invalid, or bdq:Unlikely.
bdqffdq:DataQualityReport DataQualityReport A set of bdqffdq:Assertions (bdqffdq:Measures, bdqffdq:Validations bdqffdq:Issues and bdqffdq:Amendments) that represent the output of a test run produced by a bdqffdq:Mechanism designed to assess the fitness for use of the tested data for a particular purpose as. FFU Framework: Class Veiga et al. (2017). Fitness For Use Framework
bdqffdq:DataResource DataResource Describes a data resource described in terms of a controlled vocabulary such as dwc and represents the original values of the data operated on by an assertion test (i.e. an instance of dwc:Occurrence). FFU Framework: Class Veiga et al (2017).
bdq:DefaultSourceAuthority DefaultSourceAuthority A default where a required bdq:Parameter or a bdq:sourceAuthority (q.v.) has not been provided. bdq:Parameter
bdq:defaultGeodeticDatum defaultGeodeticDatum Optionally established the default datum in a bdq:ParameterizedTest. A default datum is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter See test AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT (7498ca76-c4d4-42e2-8103-acacccbdffa7).
bdq:defaultValue defaultValue A preselected value (e.g. year, elevation) where a required bdq:Parameter value has not been provided. bdq:Parameter
bdqffdq:dimensionInContext dimensionInContext Describes the relationship between a bdqffdq:Amendment concept in the FFU Framework (needs, solutions, reports) and a bdqffdq:ContextualizedDimension. FFU Framework: ObjectProperty Veiga et al. (2017).
dwc: dwc: A namespace to indicate Darwin Core terms and which are listed in the dwcffdq:InformationElements of each Test. Data
bdq:earliestValidDate earliestValidDate Optionally establishes the earliest date in a bdq:ParameterizedTest. A default date is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter
bdq:EMPTY EMPTY A bdqffdq:InformationElement that is either not present or does not contain any characters or values other than those in the range U+0000 to U+0020. Data Note: A bdqffdq:InformationElement containing invalid characters (e.g. letters in an information element that would be expected to contain integers) or values (including string serializations of the NULL value) are NOT_EMPTY and may be separately detected.
bdqffdq:Enhancement Enhancement Describes the enhancement to the original data performed by a bdqffdq:Amendment test. For example, "Recommends valid value for taxon name in a single record". FFU Framework: Class Veiga et al (2017).
bdqffdq:enhancementInContext enhancementInContext Describes the relationship between a bdqffdq:Amendment concept in FFU Framweork (needs, solutions, reports) and a bdqffdq:ContextualizedEnhancement. FFU Framework: Object Property Veiga et al. (2017)
EPSG: EPSG A pseudo-namespace referenced in dwc:datum to indicate the EPSG API where the numeric value following the colon is used as the search key. Example: EPSG:4326. Data
bdq:Examples Examples Provide one pass (i.e. COMPLIANT) example and one fail (NON_COMPLIANT) example for each test. bdq:Parameter
bdq:ExpectedResponse ExpectedResponse bdq:ExpectedReponse is one of the properties of a bdqffdq:Specification used in the markdown of the tests in the bdq GitHub. bdq:Response
bdq:EXTERNAL_PREREQUISITES_NOT_MET EXTERNAL_PREREQUISITES_NOT_MET A bdq:Response was not generated because a bdq:sourceAuthority was not available or was off-line. If the test is run at a later time, it may produce a different result. bdq:Response.status
bdq:FILLED_IN FILLED_IN A bdqffdq:Amendment where a value has been proposed for a bdqffdq:InformationElement that has no value. bdq:Response
bdq:FOUND FOUND The value in a bdqffdq:InformationElement that matched a value in a bdq:sourceAuthority. bdq:Response Use bdq:COMPLIANT for bdq:Response.result, and include this in bdq:Response.comments or bdq:Response.qualifier.
gbif: gbif: A pseudo-namespace referenced in dwc:taxonID to indicate the GBIF API where the numeric value following the colon is used as the search key. Example: gbif:8102122. Data
bdq:GEOGRAPHY GEOGRAPHY A combination of Darwin Core administrative geography terms dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality. bdqffdq:InformationElement
bdq:geospatialLand geospatialLand Polygons derived from a union of Natural Earth vectors for Land and for Minor Islands at 1:10,000,000 resolution. bdq:Parameter See VALIDATION_COORDINATES_TERRESTRIALMARINE (b9c184ce-a859-410c-9d12-71a338200380)
bdq:GUID GUID Gobally Unique Identifier. In this document, the GUID for a test is a UUID (128-bit universally unique identifier) which identifies the test. Data GUID is intended to identify the tests for machine consumption, "Label" is used for human consumption. https://en.wikipedia.org/wiki/Universally_unique_identifier
bdqffdq:hasCriterion hasCriterion Used to link the derived concept of a bdqffdq:ContextualizedCriterion to the fundamental concept of a bdqffdq:Criterion. FFU Framework: ObjectProperty Veiga et al (2017).
bdqffdq:hasDimension hasDimension Used to link the derived concept of a bdqffdq:ContextualizedDimension to the fundamental concept of a bdqffdq:Dimension. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:hasEnhancement hasEnhancement Used to link the derived concept of a bdqffdq:ContextualizedEnhancement to the fundamental concept of a bdqffdq:Enhancement. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:hasInformationElement hasInformationElement Provides a relationship between FFDQ concepts and the information elements. For example, bdqffdq:ContextualizedCriterion uses this property along with bdqffdq:hasResourceType to define a criterion in the context of related information elements. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:hasIssue hasIssue Used to link the derived concept of a bdqffdq:ContextualizedIssue to the fundamental concept of a Problem. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:hasResourceType hasResourceType Provides additional metadata, along with the bdqffdq:InformationElements, that describes the level (bdqffdq:SingleRecord or bdqffdq:MultiRecord) at which the FFDQ concept operates. For example, a bdqffdq:enhancementInContext with resource type of bdqffdq:MultiRecord could be used to define a bdqffdq:Amendment that applies at the level of multiple record values. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:hasSpecification hasSpecification Describes the relationship between a derived FFDQ concept and the fundamental concept of a bdqffdq:Specification (technical description of a test). FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:hasStatus hasStatus Used in the bdqffdqReport concept to describe result status. For example, in the case of a bdqffdq:Validation result, values could be bdq:COMPLIANT or bdq:NON_COMPLIANT. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:Implementation Implementation The FFDQ derived concept of a bdqffdq:Implementation describes the relationship between a bdqffdq:Specification (technical description of a test) and the bdqffdq:Mechanism that implements it. FFU Framework: Class Veiga et al (2017).
bdqffdq:implementedBy implementedBy Describes the link between the bdqffdq:Implementation concept in FFDQ and the bdqffdq:Mechanism that implements some bdqffdq:Specification (also defined in bdqffdq:Implementation). FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:ImprovementTarget ImprovementTarget The bdqffdq:ImprovementTarget concept in FFDQ describes which bdqffdq:Measures and bdqqffdq:Validations are improved by some bdqffdq:Amendment. bdqffdq:ImprovementTarget includes relationships between a bdqffdq:contextualizedEnhancement (for a bdqffdq:Amendment) and one or more bdqffdq:contextualizedCriterion (link to bdqffdq:Validations) or bdqffdq:contextualizedDimension (link to bdqffdq:Measures). FFU Framework: Class Veiga et al (2017).
bdqffdq:improvedBy improvedBy Object property that describes a bdqffdq:Enhancement, as part of the bdqffdq:ImprovementTarget, that would improve data acted upon by some set of bdqffdq:Measures or bdqffdq:Validations. FFU Framework: Object Property Veiga et al. (2017).
bdq:includeEventDate bdq:includeEventDate Allows dwc:eventDate to be excluded in a bdq:ParameterizedTest. The default is to include the event date in the test, but it may be excluded to allow an identification to be prior to the event date. bdq:Parameter Used in test "VALIDATION_DATEIDENTIFIED_INRANGE" (dc8aae4b-134f-4d75-8a71-c4186239178e).
bdq:Incomplete Incomplete Where a bdqffdq:InformationElement does not contain sufficient information to satisfy the scope of the test. bdqffdq:DataQualityDimension
bdq:Inconsistent Inconsistent Where the Data Quality Dimension (q.v.): Consistency (q.v.) is not satisfied due to inconsistent values between the different Information Elements (q.v.) of a single record. bdqffdq:DataQualityDimension
bdqffdq:InformationElement InformationElement A bdqffdq:InformationElement identifies a portion of data to which a test pertains. The bdqffdq:InformationElement in FFDQ can be represented as a single or composite element that consists of one or more terms from a controlled vocabulary (fields actedUpon or consulted by an assertion test) that identifies concepts in data relevant to a use case. An abstraction or a concrete term that represents relevant content (e.g., coordinates; dwc.decimalLatitude, dwc:decimalLongitude). FFU Framework: Class For the test descriptions, bdqffdq:InformationElements are concrete Darwin Core terms, to remove ambiguity for implementors. Veiga et al (2017).
bdq:INTERNAL_PREREQUISITES_NOT_MET INTERNAL_PREREQUISITES_NOT_MET A bdq:Response.status where values of the bdqffdq:InformationElement were insufficient to run the test. If the test is run at a later time on unmodified data, it should produce the same bdq:Response. bdq:Response.status
bdq:interpretedAs interpretedAs (1) For Implementors, where Darwin Core data are serialized as strings, but the test refers to data as numeric or other non-string data type, can the string value be parsed into the target data type in the language of implementation (e.g., "1" as the integer 1), (2) matching a representation of a value unambiguously onto a controlled vocabulary (e.g., ‘WGS84’ to ’EPSG:4326’), or (3) interpreting the representation of a numeric value (e.g., a roman numeral) as a number (e.g., an integer). Data
bdq:Invalid Invalid Where the bdqffdq:DataQualityDimension: bdq:Conformance is not satisfied due to bdqffdq:InformationElements containing non-standard values. bdqffdq:DataQualityDimension
bdqffdq:Issue Issue A Data Quality needs level concept that flags issue or problems with the data. In the context of the tests, bdqffdq:Issue(s) are all either bdq:POTENTIAL_ISSUE, bdq:IS_ISSUE where potential problems are flagged and may need examination by the user to determine if data have quality for their use; or bdq:NOT_ISSUE. FFU Framework: Class Veiga et al (2017).
bdqffdq:issueInContext issueInContext Describes the relationship between a bdqffdq:Issue concept in FFU Framework (needs, solutions, reports) and a bdqffdq:ContextualizedIssue. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:IssueMethod IssueMethod A Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Issue in the context of bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements. FFU Framework: Class In Veiga et al. (2017) this was treated as "ProblemMethod"
bdqffdq:IssuePolicy IssuePolicy A Data Quality needs level concept that describes how some bdqffdq:contextualizedIssue relates to a bdqffdq:UseCase. This relationship defines which bdqffdq:Issues are supported by a given bdqffdq:UseCase. FFU Framework: Class In Veiga et al. (2017) this was treated as "ProblemPolicy"
bdqffdq:issueProperties issueProperties Sub properties of bdqffdq:ObjectProperties that apply to bdqffdq:Issue concepts such as bdqffdq:IssuePolicy (DQ needs), bdqffdq:IssueMethod (DQ solutions) and bdqffdq:Issue (DQ reports). FFU Framework: Object Property In Veiga et al. (2017) treated as "ProblemProperties"
bdqffdq:IssuesReport IssuesReport A Data Quality report level concept that results from a bdqffdq:Issue that flagged a problem in a test as bdq:IS_ISSUE, bdq:POTENTIAL_ISSUE or bdq:NOT_ISSUE. FFU Framework: Class Veiga et al. (2017).
bdq:IS_ISSUE IS_ISSUE A bdq:Response.result for a bdqffdq:Issue that flags where the data do not have sufficient quality for a use. bdq:Response.result
rdfs:label label See: https://www.w3.org/TR/rdf-schema/#ch_label RDF representations skos:preferredLabel/skos:label may be prefered.
bdq:latestValidDate latestValidDate Optionally establishes the latest date in a bdq:ParameterizedTest. A default date is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter
bdq:Likeliness Likeliness The likelihood of Darwin Core Term(s) having true or expected values. bdqffdq:DataQualityDimension Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:LineNumber LineNumber The sequence number of the data record in the bdq:ValidationData bdq:ValidationData
bdq:LineForTest LineForTest A local to bdq:ValidationData identifier for test records within one test bdq:ValiationData
bdq:maximumValidDepthInMeters maximumValidDepthInMeters Optionally establishes the maximum depth in a bdq:ParameterizedTest. A default depth is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter
bdq:maximumValidElevationInMeters maximumValidElevationInMeters Optionally establishes the highest elevation in a bdq:ParameterizedTest. A default elevation is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter
bdqffdq:Measure Measure A Data Quality needs level concept that describes a run of a test that performs a measurement according to some data quality dimension. In FFDQ, the Measure concept consists of a run result of COMPLETE or NOT_COMPLETE, a value of the measurement (i.e. a measure of dwc:eventDate duration in seconds) or counts of the number of tests from a run where bdq:Response.result was bdq:NOT_COMPLIANT, or bdq:PREREQUISITES_NOT_MET in bdqffdq:Validation tests, or was bdq:AMENDED in bdqffdq:Amendment tests. FFU Framework: Class Veiga et al. (2017).
bdqffdq:MeasurementMethod MeasurementMethod A Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Dimension in the context of bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements. FFU Framework: Class Veiga et al (2017).
bdqffdq:MeasurementPolicy MeasurementPolicy A Data Quality needs level concept that describes how some bdqffdq:contextualizedDimension relates to a bdqffdq:UseCase. This relationship defines which bdqffdq:Measures are supported by a given bdqffdq:UseCase. FFU Framework: Class Veiga et al (2017).
bdqffdq:measurementProperties measurementProperties Sub properties of bdqffdq:ObjectProperties that apply to measurement concepts such as bdqffdq:MeasurementPolicy (DQ needs), bdqffdq:MeasurementMethod (DQ solutions) and bdqffdq:Measure (DQ reports). FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:MeasurementReport MeasurementReport A Data Quality report level concept that describes the results of a run of a test that performs a measurement according to some data quality dimension. FFU Framework: Class Veiga et al. (2017).
bdqffdq:Mechanism Mechanism The FFDQ concept of bdqffdq:Mechanism describes the entity that performs an assertion test (code, external service, actor, etc.). Tied to a bdqffdq:Specification via the concept of a bdqffdq:Implementation. FFU Framework: Class Veiga et al (2017).
bdq:minimumValidDepthInMeters minimumValidDepthInMeters Optionally establishes the minimum depth in a bdq:ParameterizedTest. A default depth is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter
bdq:minimumValidElevationInMeters minimumValidElevationInMeters Optionally establishes the lowest elevation in a bdq:ParameterizedTest. A default elevation is supplied in cases where a bdq:Parameter is not set at the time the test is run. bdq:Parameter
bdqffdq:MultiRecord MultiRecord A data set composed of one or more bdqffdq:SingleRecords. FFU Framework: namedIndividual Veiga et al. (2017).
non-printing characters non-printing characters ASCII 0-32 and 127 decimal. Non printing characters or formatting marks that are not displayed at printing. These may include pilcrow, space, non-breaking space, tab character. etc. For the purposes of the tests they are treated as bdq:EMPTY. Data
bdq:NOT_AMENDED NOT_AMENDED A bdq:Result.status that indicates that a bdq:Response for a bdqffdq:Amendment proposed no change. bdq:Response.status
bdq:NOT_COMPLETE NOT_COMPLETE An assertion of a bdqffdq:Measure on a bdqffdq:MultiRecord where not all the bdqffdq:Validation bdq:Response.result from all included records in the dataset are bdq:COMPLIANT. bdq:Response.result The scope in the Fitness of Use Framework (Veiga et al. (20117) is broader.
bdq:NOT_COMPLIANT NOT_COMPLIANT A bdq:Response.result of a bdqffdq:Validation where the data do not conform to the Test bdqffdq:Criterion. Result.status
bdq:NOT_ISSUE NOT_ISSUE The bdq:Response of a test of type bdqffdq:Issue where no potential problems were detected. bdq:Response.result
bdq:NOTEMPTY NOTEMPTY The value of a bdqffdq:InformationElement that is present and has content (cf. bdq:EMPTY) Data
null null A value that is used in some databases to signify that a value is unknown or missing. It may be represented in serializations by "NULL", "Null", "null". "/n", "9999", etc. These should be treated as bdq:NOTEMPTY. Data
bdq:OUTOFRANGE OUTOFRANGE The value in a bdqffdq:InformationElement that is outside an acceptable range for that bdqffdq:InformationElement. bdq:Response Use in bdq:Response.qualifier or bdq:Response.comment.
bdq:Parameter Parameter A value provided to a test that changes the behavior of a test to fit a particular user need within the scope of the test. Either 1) a link to a bdq:sourceAuthority to find matching values, or 2) a value used to define limits for a bdqffdq:InformationElement. Data
bdq:paramaterizedTest paramaterizedTest A test that allows a bdq:Parameter to be set prior to the test being run. Where a bdq:Parameter value has not been provided, a default is specified within the test. Test
bdq:POLYNOMIAL POLYNOMIAL Represents a combination of the Darwin Core terms dwc:genericName, dwc:specificEpithet, dwc:infraspecificEpithet. bdqffdq:InformationElement See test "VALIDATION_POLYNOMIAL_CONSISTENT" (17f03f1f-f74d-40c0-8071-2927cfc9487b)
bdq:POTENTIAL_ISSUE POTENTIAL_ISSUE A bdq:Response.result for a bdqffdq:Issue that flags where the data may not have sufficient quality for a use. See also bdq:IS_ISSUE and bdq:NOT_ISSUE. The user will need to evaluate if the data is fit for their particular use or not. bdq:Response.result
bdq:PRECISIONINSECONDS PRECISIONINSECONDS The length of the period of an event in seconds. bdq:Response This is description of the bdq:Response.result from this bdqffdq:Measure, where the result is a numeric value in seconds. See Test "MEASURE_EVENTDATE_DURATIONINSECONDS" (56b6c695-adf1-418e-95d2-da04cad7be53).
bdq:PREREQUISITESNOTMET PREREQUISITESNOTMET A test of type bdqffdq:Measure that counts the number of tests of type bdqffdq:Validation that did not run due to one or more prerequisites not being met (e.g. bdq:INTERNAL_PREREQUISITES_NOTMET and bdq:EXTERNAL_PREREQUISITES_NOTMET) bdq:Response See test "MEASURE_VALIDATIONTESTS_PREREQUISITESNOTMET" (49a94636-a562-4e6b-803c-665c80628a3d).
bdqffdq:Profile Profile a Data Quality needs level concept describing the bdqffdq:UseCases that make up some data quality operation such as the behavior of a single actor or workflow producing the relevant bdqffdq:Assertions. FFU Framework: Class Veiga et al. (2017).
bdq:PROPOSED PROPOSED A test of type bdqffdq:Measure that pertains to a bdqffdq:Amendment where an action to modify a value in some way through a change or addition is recommended. bdq:Response Example see test "MEASURE_AMENDMENTS_PROPOSED" (03049fe5-a575-404f-b564-ae63f5a1cf8b).
bdq:Record-Management Record-Management Management of the quality of biodiversity data records (see examples in Rees ER & Nicholls M (2020) Data Quality Use Case Study Results https://doi.org/10.3897/biss.4.50889.suppl2). bdqffdq:UseCase
bdq:Reliability Reliability Measure of how the data values agree with an identified source of truth. The degree to which data correctly describes the truth (object, event or any abstract or real 'thing'). bdqffdq:DataQualityDimension Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:Resolution Resolution Refers to the data having sufficiently detailed information. Measure of the granularity of the data, or the smallest measurable increment. bdqffdq:DataQualityDimension Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:Response Response The report from a single execution of a single test, consisting of bdq:Response.status, bdq:Response.result, bdq:Response.comment, and optionally, bdq:Response.qualifier. bdq:Response Parent of RESULT and RESULT_STATUS in the Fitness for Use Framework (Viega et al. 2017).
bdq:Response.comment Response.comment Human readable interpretation of the results of the test. Response
bdq:Response.qualifier Response.qualifier Additional structured information that qualifies the bdq:Response, intended as an extension point for uncertainty. bdq:Response
bdq:Response.result Response.result The element in a bdq:Response containing the value returned by the particular test (bdqffdq:Validation, bdqffdq:Amendment, bdqffdq:Measure, or bdqffdq:Issue) bdq:Response
bdq:Response.status Response.status A metadata element in a bdq:Response indicating whether a particular test (bdqffdq:Validation, bdqffdq:Amendment, bdqffdq:Measure, or bdqffdq:Issue) was able to be performed or not. bdq:Response
bdqffdq:ResourceType ResourceType In FFDQ the concept of bdqffdq:ResourceType has instances for bdqffdq:SingleRecord or bdqffdq:MultiRecord. FFU Framework: Class Veiga et al. (2017).
bdqffdq:Result Result The report bdqffdq:Result concept in FFDQ is represented as a value or a bdqffdq:ResultStatus for bdqffdq:Measures, just a bdqffdq:ResultStatus for bdqffdq:Validations and a bdqffdq:ResultStatus as well as values for changes proposed by bdqffdq:Amendments. FFU Framework: Class Veiga et al. (2017).
bdqffdq:ResultStatus ResultStatus Depending on the assertion type would have values of bdq:COMPLIANT or bdq:NOT_COMPLIANT for a bdqffdq:Validation, bdq:COMPLETE or bdq:NOT_COMPLETE for a bdqffdq:Measure, bdq:AMENDED, bdq:FILLED_IN, bdq:TRANSPOSED, bdq:NOT_AMENDED for a bdqffdq:Amendment and bdq:IS_ISSUE, bdq:POTENTIAL_ISSUE or bdq:NOT_ISSUE for a bdqffdq:Issue FFU Framework: Class Veiga et al. (2017). Note that a separate concept describes the resultstate with values of bdq:INTERNAL_PREREQUISITES_NOT_MET and bdq:EXTERNAL_PREREQUISITES_NOT_MET.
Roman numerals Roman numerals Roman numerals are interpreted as the equivalent integer for months (e.g. "X" as "10") in appropriate tests. Roman numerals may not be unambiguously interpreted for other Darwin Core terms such as dwc:day or in text fields as they may mean unknown or something else entirely. Data
bdq:RUN_HAS_RESULT RUN_HAS_RESULT A bdq:Response.status that implies that a result was correctly generated. bdq:Response.status Applies to bdqffdq:Validations, bdqfdfq:Measures, and bdqffdq:Issues, but not bdqffdq:Amendments. See Fitness for Use Framework definition in Need link to OWL Document @chicoreus. See also bdq:INTERNAL_PREREQUISITES_NOT_MET and bdq:EXTERNAL_PREREQUISITES_NOT_MET
bdqffdq:SingleRecord SingleRecord A record from a dataset without dependencies on any other record. FFU Framework: namedIndividual Veiga et al. (2017). Note that all the current tests are run on a bdqffdq:SingleRecord, of Darwin Core data, and not designed to be run across a bdqffdq:MultiRecord, except for bdqffdq:MultiRecord bdqffdq:Measures.
bdq:sourceAuthority sourceAuthority An authority using the "bdq" namespace that provides a reference for values required for a test evaluation. Where the test is a bdq:ParameterizedTest a bdq:defaultSourceAuthority ("bdq:sourceAuthority default = xxx") is specified. bdq:Parameter
bdq:spatialBufferInMeters spatialBufferInMeters A buffer in meters from a polygon (geopolitical boundary, coastline, etc.). bdq:Parameter
bdq:Spatial-Temporal Patterns Spatial-Temporal Patterns Research uses for biodiversity occurrence data where 1) the information elements concern what organism occurred where and when and 2) that are used for analysis of spatial and/or temporal patterns of biodiversity (see examples in Rees ER & Nicholls M (2020) Data Quality Use Case Study Results https://doi.org/10.3897/biss.4.50889.suppl2). bdqffdq:UseCase
bdqffdq:Specification Specification A technical description of the performed test upon which a bdqffdq:Implementation could be made. bdq:Response
bdq:STANDARD STANDARD A bdqffdq:Amendment where a value in a bdqffdq:InformationElement is proposed from a bdq:sourceAuthority. bdq:Response Use in bdq:Response.qualifier or bdq:Response.comment.
bdq:STANDARDIZED STANDARDIZED A bdqffdq:Amendment where a bdq:STANDARD value for a bdqffdq:InformationElement is proposed. bdq:Response Use bdq:AMENDED as the bdq:Response.status, report bdq:STANDARDIZED in a bdq:Response.qualifier or in a bdq:Response.comment.
bdq:targetCRS targetCRS The Coordinate Reference System (CRS) used as the output when converting coordinates from one CRS to another. The default is EPSG:4326. bdq:Parameter Used in the test AMENDMENT_COORDINATES_CONVERTED (620749b9-7d9c-4890-97d2-be3d1cde6da8).
bdqffdq:targetedCriterion targetedCriterion The bdffdq:Criterion targeted by some enhancement via the bdqffdq:ImprovementTarget object. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:targetedDimension targetedDimension The bdqffdq:Dimension targeted by some enhancement via the bdqffdq:ImprovementTarget object. FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:targetedIssue targetedIssue The bdqffdq:Issue targeted by some problem via the bdqffdq:ImprovementTarget object. FFU Framework: Object Property Veiga et al. (2017).
bdq:Taxon-Management Taxon-Management Management of the quality of taxonomic names (see examples in Rees ER & Nicholls M (2020) Data Quality Use Case Study Results https://doi.org/10.3897/biss.4.50889.suppl2). bdqffdq:UseCase
bdq:taxonomyIsMarine taxonomyIsMarine Marine/non-marine status obtained from the World Register of Marine Species (WORMS) database bdq:Parameter See VALIDATION_COORDINATES_TERRESTRIALMARINE (b9c184ce-a859-410c-9d12-71a338200380).
bdq:TERRESTRIALMARINE TERRESTRIALMARINE A terrestrial taxon that has geographic coordinates that fall within terrestrial boundaries; or a marine taxon that has geographic coordinates that fall within marine boundaries. bdq:Response Use bdq:AMENDED as the bdq:Response.status, report bdq:TERRESTRIALMARINE in a bdq:Response.qualifier or in a bdq:Response.comment. See test "VALIDATION_COORDINATES_TERRESTRIALMARINE" (b9c184ce-a859-410c-9d12-71a338200380).
bdq:TestField testFields Column heading in the markdown of the tests in the bdq GitHub that list all the normative and informative metadata elements that describe a Data Quality Test Test
bdq:TestPrerequisite TestPrerequisite Conditions that must be met for a test to be run (e.g., fields having values, tests that need to be run before the current test, availability of a bdq:sourceAuthority) Test See for example, INTERNAL_PREREQUISTES_NOT-MET (q.v.) and EXTERNAL_PREREQUISITES_NOT_MET (q.v.).
bdq:TestType TestType There are four types of tests, viz. bdqffdq:Validation, bdqffdq:Amendment, bdqffdq:Issue, and bdqffdq:Measure. Test  
bdq:TRANSPOSED TRANSPOSED The sign and/or value of one or more bdqffdq:InformationElements were swapped. bdq:Response Use bdq:AMENDED as the bdq:Response.status, report bdq:TRANSPOSED in a bdq:Response.qualifier or in a bdq:Response.comment. See Test "AMENDMENT_COORDINATES_TRANSPOSED" (f2b4a50a-6b2f-4930-b9df-da87b6a21082).
bdq:Unlikely Unlikely The bdqffdq:DataQualityDimension: bdq:Likeliness is not satisfied due to the bdq:InformationElements containing a value that is not likely to occur (for example where the geographic coordinates are "0", "0"). bdqffdq:DataQualityDimension Needs a bdq:Response.qualifier (q.v.) response for the uncertainty.
bdqffdq:UseCase UseCase The bdqffdq:UseCase concept in FFDQ describes some data quality control use case. The bdqffdq:Amendment, bdqffdq:Measure and bdqffdq:Validation policies that make up a bdqffdq:UseCase define which bdqffdq:Assertions cover a given bdqffdq:UseCase. FFU Framework: Class An example of a bdqffdq:UseCase could be "Check for internal consistency of dates", with bdqffdq:Validation policies for checking consistency between atomic date fields and a bdqffdq:Amendment such as "eventDate filled in from verbatim". Veiga et al. (2017).
bdqffdq:Validation Validation A Data Quality needs level concept that describes a run of a test for validity. The bdqffdq:Validation concept in the Tests consists of a run with a bdq:Response:result of bdq:COMPLIANT or bdq:NOT_COMPLIANT and a bdqffdq:Criterion that describes the conditions for validity that result in a status of bdq:COMPLIANT. FFU Framework: Class Veiga et al. (2017).
bdq:ValidationData ValidationData Test data set established for testing Test Implementations Data
bdqffdq:ValidationMethod ValidationMethod TA Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Criterion in the context of a bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements. FFU Framework: Class Veiga et al. (2017).
bdqffdq:ValidationPolicy ValidationPolicy A Data Quality needs level concept that describes how some bdqffdq:contextualizedCriterion relates to a bdqffdq:UseCase. This relationship defines which validations are supported by a given bdqffdq:UseCase. FFU Framework: Class Veiga et al. (2017).
bdqffdq:validationProperties validationProperties Sub properties of bdqffdq:ObjectProperties that apply to validation concepts such as bdqffdq:ValidationPolicy (DQ needs), bdqfdq:ValidationMethod (DQ solutions) and bdqffdq:Validation (DQ reports). FFU Framework: Object Property Veiga et al. (2017).
bdqffdq:ValidationReport ValidationReport A Data Quality report level concept that reports the results of a run of a bdqffdq:Validation test on some data. FFU Framework: Class Veiga et al. (2017).
bdq:VERBATIM VERBATIM An original value. bdqffdq:InformationElement
white space white space 1) A field that only includes white space (blanks) is treated as bdq:EMPTY (q.v.). 2) In bdqffdq:Validation tests (q.v.) that require the looking up of a bdq:sourceAuthority, leading and/or trailing white space will cause the test to fail as no preprocessing is carried out on the data. These leading and trailing white spaces may be stripped out in a subsequent bdqffdq:Amendment and thus pass when the bdqffdq:Validation test is run again. Data
bdq:YEARMONTHDAY YEARMONTHDAY Represents a combination of the Darwin Core terms dwc:year, dwc:month, dwc:day. bdqffdq:InformationElement
bdq:YEARSTARTDAYOFYEARENDDAYOFYEAR YEARSTARTDAYOFYEARENDDAYOFYEAR Represents a combination of the Darwin Core terms dwc:year, dwc:startDayOfYear, dwc:endDayofYear. bdqffdq:InformationElement

Supplement: GitHub Label Terms These are terms that are outside the Standard but that have been used as either GitHub Labels or TestFields in the BDQ GitHub

namespace:Term Label Definition Context Comment
bdqtag:Amendment Amendment A label to indicate a test of type AMENDMENT which may propose a change or addition to at least one Darwin Core term that is intended to improve one or more components of the quality of the record. GitHub Label See bdqffdq:Amendment
bdqtag:CORE CORE Tests for evaluating biodiversity data quality as represented by the values of Darwin Core terms. CORE tests address identified user needs, are widely applicable, informative, unambiguous, well defined, and straight forward to implement. GitHub Label
bdqTestField:Darwin Core Class Darwin Core Class The Information Element in the original terms of the framework, the general sort of information this test operates on. TestField
bdqTestField:Data Quality Dimension Data Quality Dimension The data quality dimension for this test. See bbqffdq:DataQualityDimension. TestField
bdqTestField:Description Description A non-technical description of what the test does, intended for consumers of data quality reports in concert with the bdq:Response.comment. TestField
bdqtag:DO NOT IMPLEMENT DO NOT IMPLEMENT Tests that are not CORE (cf. bdqtag:CORE) and not recommended to be implemented with the current level of understanding for one or more reasons: Available vocabularies are ambiguous; the test is too complex to implement concisely; implementation is expected to lead to ambiguous or inaccurate results. GitHub Label
bdqTestField:Example Implementations (Mechanisms) Example Implementations (Mechanisms) Known Mechanisms with implementations of the test. TestField
bdqTestField:Examples Examples A ’pass’ and a ‘fail’ example of test data. All examples listed are present in the the validation data suite. TestField
bdqTestField:Expected Response Expected Response The specification for implementors describing the expected behavior of the test. See bdqffdq:Specification TestField = bdqffdq:Specification
bdqTestField:GUID GUID see bdq:GUID TestField
bdqtag:Immature/Incomplete Immature/Incomplete Tests where substantial work is needed to develop the specification to the point where the test can be reliably and usefully implemented. This may indicate work that is wholly internal to the test specification such as developing a consistent Expected Response, or may indicate that external work is needed to develop an agreed vocabulary for values of the tested term. An immature/incomplete test may be made CORE, Supplementary, or DO NOT IMPLEMENT when relevant criteria are satisfied.
bdqTestField:Information Elements Acted Upon Information Elements Acted Upon A list of the specific Darwin Core terms that are the focus of a test. TestField
bdqTestField:Information Elements Consulted Information Elements Consulted AA list of Darwin Core terms that are consulted in the evaluation of the Information Elements ActedUpon. TestField
bdqtag:ISO/DCMI STANDARD ISO/DCMI STANDARD A reference to either an ISO (International Organization for Standardization) Standard or a DCMI (Dublin Core Metadata Initiative) Standard GitHub Label
bdqtag:Issue Issue A label to indicate a test of type ISSUE, where potential problems are flagged and may need examination by the user to determine if data have quality for their use. GitHub Label see bdqffdq:Issue
bdqTestField:Label Label A human readable label identifying the test. The labels largely follow the pattern TYPE_INFORMATIONELEMENT_STATUS. TestField cf. rdfs:label
bdqTestField:Link to Specification Source Code Link to Specification Source Code A link to code that implements the test. TestField
bdqtag:Measure Measure A label to indicate a test of type MEASURE that performs a measurement according to some data quality dimension. GitHub Label See bdqffdq:Measure
bdqtag:NAME NAME A label to indicate that the test is related to Darwin Core terms in the dwc:Taxon Class. GitHub Label
bdqtag:NEEDS WORK NEEDS WORK A label that indicates that an issue (Test) requires more work before finalising. GitHub Label
bdqTestField:Notes Notes Additional, non-normative comments that the Task Group believed necessary for an accurate understanding of the test or issues that implementers needed to be aware of. TestField
bdqtag:OTHER OTHER A label to indicate that the test is related to Darwin Core terms other than Classes dwc:Taxon, dwc:Location or dwc:Event. GitHub Label
bdqtag:Parameterized Parameterized A label for a test that requires a bdq:Parameter to be set prior to a bdq:parameterizedTest being run. GitHub Label
bdqTestField"Parameter(s) Parameter(s) Any parameters that change the behavior of the test for a subset of users with special data quality needs within the domain. TestField
bdqTestField:References References A list of references pertinent to the test. TestField
bdqTestField:Source Source The origin of the concept of the test. TestField
bdqTestField:Source Authority Source Authority A reference to an external (non-Darwin Core) authority required for the test. See bdq:sourceAuthority TestField
bdqtag:SPACE SPACE A label to indicate that the test is related to Darwin Core terms in the dwc:Location Class. GitHub Label
bdqTestField:Specification Last Updated Specification Last Updated The last date a change was made to a test that affects the operation of the test. TestField
bdqtag:Supplementary Supplementary Tests regarded as not CORE (cf. bdqtag:CORE) because of one or more reasons: Not widely applicable; not clearly matched to an identified data quality need; not informative concerning the 'quality' or lack of quality of the data; likely to return a high percentage of either bdq:NOT_COMPLIANT or bdq:POTENTIAL_ISSUE records. A Supplementary test MAY be implemented in a local implementation if a suitable use case exists. GitHub Label A Supplementary test may be made CORE at a later time.
bdqTestField:Term-Actions Term-Actions Equivalent to the bdqTestField:Label without the leading Test Type. TestField
bdqtag:Test Test Tests created by TG2, either CORE, Immature/Incomplete, Supplementary, or DO NOT IMPLEMENT. GitHub Label
bdqTestField:Test Type Test Type The Type of assertion that the test produces, Measure, Validation, Amendment, Issue. TestField
bdqtag:TG1 TG1 Issues pertinent to Task Group 1 (Framework on Data Quality) of the TDWG Data Quality Interest Group. GitHub Label
bdqtag:TG2 TG2 Issues including Tests, developed by, or pertinent to Task Group 2 (Data Quality Tests and Assertions) of the TDWG Data Quality Interest Group. GitHub Label
bdqtag:TG3 TG3 Issues pertinent to Task Group 3 (Data Quality Use Cases) of the TDWG Data Quality Interest Group. GitHub Label
bdqtag:TG4 TG4 Issues pertinent to Task Group 4 (Best Practices for Development of Vocabularies of Value) of the TDWG Data Quality Interest Group. GitHub Label
bdqtag:TIME TIME A label to indicate that the test is related to Darwin Core terms in the dwc:Event Class. GitHub Label
bdqtag:Validation Validation A label to indicate a test of type VALIDATION that describes a run of a test for validity against a set of criteria. GitHub Label See bdqffdq:Validation
bdqtag:VOCABULARY VOCABULARY A label to indicate that a bdqlabel:Test requires a Vocabulary GitHub Label
chicoreus commented 5 years ago

Much (or all) of the vocabulary will come out of the framework as a technical specification, probably with additional supporting vocabularies (such as for values for data quality dimension). There is still a need to express the tests themselves as a formal specification (s.l.) and move this towards a TDWG standard.

ArthurChapman commented 5 years ago

There may (probably) be terms associated with the Tests and Assertions beyond the Framework. The Framework's terms are broader than just Darwin Core - the TG2 tests need to define some terms that are outside the Framework (CORE is one that comes to mind. Whether it makes sense or not to expand the Framework terms to cover these is probably worth discussing.

tucotuco commented 5 years ago

I propose that the framework should have a distinct vocabulary product consisting of the framework terms and their controlled vocabularies of values. I propose that there we a task group spawned specifically to create this. Tests and assertions rely on these for rigorous definition, so to me it has highest priority as a new vocabulary.

chicoreus commented 5 years ago

@tucotuco sounds like a deliverable from TG1.

saraiva-usp commented 5 years ago

Agree!

ArthurChapman commented 5 years ago

Are there terms that we may be using in TG2 that are outside of the Framework such that we need a separate vocabulary (but where most terms link to the Framework, or to Darwin Core)? I will go through the Tests and pull out a list of terms for which I think may need definitions.

Tasilee commented 5 years ago

We do have terms in TG2 that are beyond the Framework. I'll work with @ArthurChapman to generate the list.

ArthurChapman commented 5 years ago

In the PSSR-CORE (Citizen Science) document from 2017 (https://www.wilsoncenter.org/sites/default/files/wilson_171204_meta_data_f2.pdf)

One of the tasks mentioned in that report is 1 Understand and develop a common vocabulary for discussing the range of data quality practices in citizen science.

Peter Brenton is going to send me the name of someone from that working group with whom we should liaise

ArthurChapman commented 5 years ago

@Tasilee @pzermoglio and others. What columns do we require in our draft dq vocabulary? Some initial suggestions

Term | Definition | Source | Reference | Link | GUID |

chicoreus commented 5 years ago

Let me suggest the list of columns found in the header column in this Audubon Core source document: https://github.com/tdwg/rs.tdwg.org/blob/master/audubon/audubon-column-mappings.csv

In particular:

label rdfs_comment dcterms_description term_localName term_isDefinedBy term_created term_modified
(=skos:preferedLabel (en)) (=skos_definition) (=dcterms:description) (with term_isDefinedBy forms the guid) (=dcterms:isPartOf) (=dcterms:created) (=dcterms:modified)

We should also check skos for appropriate skos terms for source, reference, and link.

ArthurChapman commented 5 years ago

Thanks @chicoreus - I will look at that. We have a few different processes - the DQ Vocabulary - and that will depend a lot on what @pzermoglio comes up with. TG1 - Vocabulary will form a major part of the Vocabulary. I am looking at extracting the terms from the tests and just want to make sure we capture what we need at this stage so that we can then add the terms to the main Vocabulary we develop and not have to revisit things later.

Tasilee commented 5 years ago

Those columns of @ArthurChapman are 95% the table from my keynote :) and ah yes, there is SKOS

tucotuco commented 5 years ago

For Darwin Core, the full set of columns to manage the term definitions, usages, and examples is:

iri,label,definition,comments,examples,organized_in,issued,status,replaces,rdf_type,term_iri,abcd_equivalence,flags

Tasilee commented 5 years ago

From @tucotuco: "To me it is clear that the Framework will result in a vocabulary that should be made into a standard. To me this is separate from a possible standard arising from the tests and assertions. These are two distinct products to me, with the latter relying on the former, thus increasing the priority of the former."

"Steve Baskauf clarified that the TDWG Standards Documentation Standard (SDS; https://www.tdwg.org/standards/sds/, in its single document, the "TDWG Standards Documentation Specification") describes how to create Data standards (for Vocabularies) as well as Best Current Practices documents. The Vocabularies of Values Best Current Practices Document must conform with that document, just as any vocabularies of values must also conform to the specifications set out in the SDS. The DQIG believes that a Vocabularies of Values Best Current Practices document is needed to provide more specific and common guidance on vocabularies of values construction and maintenance - for example, guidance on the type of vocabulary to use (Thesaurus, Vocabulary, Dictionary, Ontology, etc.), and how to deal with synonymy, multiple languages, etc."

I suggest then a rename of this issue to "TG2-" and tags. Alan, Miles...can create a separate issue. :)

ArthurChapman commented 5 years ago

I have added the terms above that I feel may need defining for the Tests. I have attempted to put in a definition - although some are still blank. If you have any suggestions, comments, etc. please comment.

The Terms in UPPER CASE - are terms used in the Title of the Test

ArthurChapman commented 5 years ago

I am not sure that we need to define CORE. It is a working term and the criteria by which tests are included will, I am sure, be described in the Introduction to the Standard.I suggest we delete it from the Vocabulary

ArthurChapman commented 5 years ago

Does "Output Type" come from the Framework @allankv ? If not I suggest we change the term in the tests as it doesn't make a lot of sense when you look at the required values (Validation, Amendment, Notification and Measure)

chicoreus commented 5 years ago

@ArthurChapman I think "output type" is just mapping to owl:type, see: https://github.com/kurator-org/kurator-ffdq/blob/master/competencyquestions/rdf/ffdq.owl In the conversion to a csv file, it gets renamed "Type". No need to change the value in the current markdown tables in the issues, but also no need to define Type in a tdwg namespace.

@ArthurChapman "CORE" is at least the label of the Data Quality Profile that comprises the suite of tests produced by TG2. but I think more likely, it is an instance of a use case that helps identify the elements of the validation policy, measurement policy, and enhancement policy that form the data quality profile for the tests. See the Data Quality Needs section of the framework: https://github.com/tdwg/bdq/wiki/TG1-Framework-Cheat-Sheet

ArthurChapman commented 5 years ago

@chicoreus. You have added Information Element to, for example, COORDINATES - but this isn't an Information Element but a term we use in the title to indicate the Target. Sometimes the Target is a dwc:term but not always. so I don't think we should define it as such.

ArthurChapman commented 5 years ago

@chicoreus There are several like that. COORDINATES, GEOGRAPHY, POLYNOMIAL, YEARMONTHDAY, YEARENDDAYOFYEARSTARTDAYOFYEAR. Generally our test names go VALIDATION_TARGET_ACTION etc. where target is often an abbreviation of an Information Element (like YEAR, etc.), but in the cases above, and in several others - e.g. MAXELEVATION etc. they refer to one Information Element (e.g. MAXELEVATION is defined by - see the spreadsheet - by saying see dwc:maxElevationInMeters etc. The ones listed at the start of the Comment - COORDINATES, etc. can only be defined as a combination of several information elements.

The current definition of Information Elements given in the Framework do include a mix of things (Coordinates, Date, Time, Species, Specimen, Observation). Doing this exercise (TG2 and the Vocabulary), I am not sure the Framework Definition is a good one - or we have been a little inconsistent in our use of it to refer only to Darwin Core or Dublin Core Terms (except for bdq:isMarine - see below)

The only Information Element we don't have defined (through Darwin Core or Dublin Core) is bdq:isMarine(#51 ) and to would be nice to be able to describe that test without having to create a new Information Element.

Now I see two options:

  1. the Framework redefine the Information Element and perhaps find a new term for the more general terms like Coordinates, Polynomial, etc.
  2. We change the "Information Element" in the tests to something like "terms"

In any case I think we need to look at the examples given in the Framework for IE - and rather than species we use Polynomial, etc.

ArthurChapman commented 5 years ago

@chicoreus I have added another term "Response.status" - can you please define this.

ArthurChapman commented 5 years ago

@chicoreus I have removed Output Type as you said elsewhere that when you export it to a csv. file this is changed to Type ... Thus we don't need a definition. Thanks.

ArthurChapman commented 5 years ago

Please note that the Definitions listed above are only definitions that are specific to the Tests. The full list of definitions - e.g. those that refer to other sources (Darwin Core, Dublin Core, TGN, The DQ Framework, etc.) are all included in a Spreadsheet which can be found at https://drive.google.com/open?id=1NCMFz_hBIACuuzruxo2mAfIeM_dNqwog0n2sPIf8SFw

chicoreus commented 5 years ago

@ArthurChapman regarding Response.Status. See the tables in https://github.com/tdwg/bdq/issues/142#issuecomment-376734516 and https://github.com/tdwg/bdq/issues/142#issuecomment-382475645

Regarding Information Element. COORDINATES, etc, are definitely information elements in the sense of the framework. Allan (and us in the publication on the framework) have used this form (and that specific case) of a composite information element multiple times. We've had to push hard to get the framework to accept the lists of specific terms as also being part of the definition of an information element, as well as this general label (the specific terms are necessary for implementation).

ArthurChapman commented 5 years ago

@chicoreus Then should we change the name of what we have as "Information Element" in the tests? to Terms or Term Elements perhaps?

chicoreus commented 5 years ago

@ArthurChapman I don't think so. They are both Information Elements under the framework, and then we'd have to create labels for information elements made up of single terms.

chicoreus commented 5 years ago

Perhaps clarifying more, we've got tests that take information elements composed of single darwin core terms (e.g. dwc:year), then others that take information elements composed of more than one term (yearmonthday(dwc:year,dwc:month,dwc:day), then, quite separately, we've got labels (like TIME) to categorize tests into those with information elements covering similar concepts.

ArthurChapman commented 5 years ago

Thanks @chicoreus - that is very helpful. BTW we have done away with NAME, TIME, SPACE and OTHER (except as Labels in GitHub because they can be adequately covered by the Darwin Core Classes Taxon, Event, Location, Occurrence and Record-level Terms

ArthurChapman commented 5 years ago

In discussion with @Tasilee, and in looking at our BISS paper, we have decided to change 'Action' to 'Response' as terms like 'EMPTY' 'PRECISIONINSECONDS', etc. are not Actions. Thus we now have the tests in the form of

ArthurChapman commented 5 years ago

I have modified and updated the Vocabulary for TG2. I have added some new terms, and some of the terms from the Framework (they are noted) with a link - however note that not all the Framework definitions are finalized. Terms from the Framework are in Italics in the Term column In some cases the definition, as written in the Framework, doesn't coincide 100% with the way we are using the term in the Tests, so I have modified the definition and made a note that it differs from the Framework Definition. In doing this, I think it is clear that some of the Framework definitions may need to be modified.

Please check them thoroughly, and make any comments suggestions, etc.

ArthurChapman commented 4 years ago

@chicoreus. Just trying to get my head around the definitions of the Namespace elements. I am not sure about the bdq:use.... elements e.g.

bdq:useEarliestValidDate (should this be bdq:earliestValidDate for consistency with ...ValidDepth and ...ValidElevation? or bdq:earliestDate)

bdq:useEarliestValidDate: A Parameter (q.v.) that ..... (@chicoreus - help needed)

bdq:earliestValidDate: A Parameter (q.v.) that optionally establishes the earliest date in a parameterized test. A default date is supplied in cases where a parameter is not set at the time the test is run.

chicoreus commented 4 years ago

@ArthurChapman, when a parameter such as bdq:earliestValidDate is composed with a test specification that states the parameter should be optionally applied, then another parameter is entailed to assert whether or not the earliestValidDate parameter should be applied or not.

Expectation would be in the form:

bdq:earliestValidDate=1700 bdq:useEarliestValidDate=true, test for dates back to 1700.

bdq:earliestValidDate= bdq:useEarliestValidDate=true, test for dates back to the specified default value for earliestValidDate.

bdq:earliestValidDate=1700 bdq:useEarliestValidDate=false, dates have no lower bound.

bdq:earliestValidDate= bdq:useEarliestValidDate=false, dates have no lower bound.

bdq:use{Foo} a parameter that, when equal to true, asserts that the parameter foo should be applied in the test where its application is optional, if no value is provided for the parameter foo, then the default value is applied. When the useFoo parameter has a value equal to false, then the test where the application of foo is optional does not use a given value of foo or the specified default value of foo in the test.

Note that use{Foo} parameters will only be coupled with foo parameters when a test specification asserts that the application of the foo parameter in the test is optional, e.g. a test that specifies an optional lower bound with a default lower limit may use that default lower limit, may use a specified lower limit, or may not test at all for a lower limit.

Given bdq:foo default=1, (1) where foo= and useFoo=true, then the default value of foo of 1 is used in the test. (2) where foo=2 and useFoo=true, then the provided value of foo of 2 is used in the test. (3) where foo= and useFoo=false, then foo is not tested for. (3) where foo=2 and useFoo=false, then foo is not tested for.

ArthurChapman commented 4 years ago

I have added definitions for all the bdq name space elements (bdq:...). I have also added a definition for "paramaterized test". Could you please check these definitions, before I add them to the BISS paper.

Tasilee commented 4 years ago

Nice work @ArthurChapman

Tasilee commented 2 years ago

Should we add "CHANGED" and "NOT CHANGED" as we are likely to standardize on these terms? Is there another context to use "AMENDED"? There may be a few others in the Expected Responses. I'll start checking.

chicoreus commented 2 years ago

Added NOT_AMENDED, per discussion on 2022 Feb 27.

ArthurChapman commented 2 years ago

Some TERMS that we may need to add to Glossary [NB as added, add an "x" between the square brackets]

Terms to be edited

Terms to delete?

Tasilee commented 2 years ago

Thanks Arthur

I'm unsure of the context of some of these terms. I will add discussion of the column headers of the test data worksheet to the next agenda.

Tasilee commented 2 years ago

Following up on vocabularies, we have confusion with #164 and given @chicoreus comment about 4 vocabularies required for the standard and a lack of consistent context...which often comes first in the Definition column.

May I suggest

  1. I add a new COLUMN 3 called something like "Context" with values such as "DQ-DIMENSION" and "Warning Type" that are currently embedded in the Definitions. I find the current structure messy and inconsistent. Example: Reference to FFU is too broad.

  2. We need to check that we have terms and definitions from the column headers of the various tables we use. For example, we use the term "Label" in what I would call the test specifications (the top table on the test issues), e.g., "AMENDMENT_TAXONID_FROM_TAXON" but this is not in the vocabulary. Ditto for example "Comment" in the test data worksheet. Should this be "Test.Data.Comment" or something else to make it unique? Yes, some terms may not be 'normative', but if we use them in a consistent context, it does no harm to include them here. I am happy to add in those terms with the understanding that we ALL need to review the terms and their parameters at some point when the test data is finalized.

chicoreus commented 2 years ago

On Tue, 08 Mar 2022 16:23:14 -0800 Lee Belbin @.***> wrote:

For example, we use the term "Label" in what I would call the test specifications (the top table on the test issues), e.g., "AMENDMENT_TAXONID_FROM_TAXON"

This is the rdfs:label.

See the RDF representation of the tests (I've just updated today)

431467d6-9b4b-48fa-a197-cd5379f5e889 is the identifier for a Specification, AMENDMENT_TAXONID_FROM_TAXON is it's label.

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority was not available; INTERNAL_PREREQUISITES_NOT_MET if all of dwc:kingdom, dwc:phylum, dwc:class, dwc:order, dwc:family, dwc:genus, and dwc:scientificName are EMPTY; AMENDED if a value for dwc:taxonID is unique and resolvable on the basis of the value of the lowest ranking not EMPTY taxon classification terms dwc:scientificName, dwc:scientificNameAuthorship, dwc:kingdom, dwc:phylum, dwc:class, etc.; otherwise NOT_AMENDED AMENDMENT_TAXONID_FROM_TAXON https://github.com/tdwg/bdq/blob/4519d129f6ab3e142f38526a328f6f9309bc654e/tg2/core/TG2_tests.xml#L1055 > but this is not in the vocabulary. > Ditto for example "Comment" in the test data worksheet. This is Response.comment (or whatever we are calling the result/response/returnvalue thing) - it has three requried parts, and we've put an optional fourth part on the table. The comment is required, the content in it does not have to match what we have in the test data. Response.status Response.value/result Response.comment -Paul -- Paul J. Morris Biodiversity Informatics Manager Museum of Comparative Zoölogy, Harvard University ***@***.*** AA3SD PGP public key available
ArthurChapman commented 2 years ago

I have added "response.comment" into the list above of terms that need to be added.

ArthurChapman commented 2 years ago

I have also added "rdfs:label" into the terms that need to be added above

Tasilee commented 2 years ago

OK. I've done a first pass through the table and naturally, there are plenty of things to discuss. The new layout seems far better to me.

I have added "Response.comment" and "rdfs:label"

ArthurChapman commented 2 years ago

I've ticked those off in the list of terms to be added above.

ArthurChapman commented 2 years ago

I have added "null", "NOT_COMPLIANT" and "RUN_HAS_RESULT" to the vocabulary.

ArthurChapman commented 2 years ago

@Tasilee Note that we have a file with lots more terms, many not included within this file but with definitions. See https://docs.google.com/spreadsheets/d/1NCMFz_hBIACuuzruxo2mAfIeM_dNqwog0n2sPIf8SFw/edit#gid=1530751621

ArthurChapman commented 2 years ago

I have added "non-printing characters" and attempted to define them. Also "RUN_HAS_RESULT"

ArthurChapman commented 2 years ago

Following email discussions with respect to Test #101 whether the values in a primary Darwin Core term are Consistent with values in the values in the atomic terms (e.g. dwc:scientificName with dwc:genus, dwc:specificEpithet and dwc:infraspecificEpithet) when one or more of the atomic terms are EMPTY. It was agreed that for the purposes of the tests if one or more of the atomic terms is empty and the others are consistent then the the test should return COMPLIANT for consistency (for example in if dwc:infraspecificEpithet is empty but that dwc:genus and dwc:specifiecEpithet are consistent with the values in dwc:scientificName - then it is COMPLIANT). Thus - I have changed the definition of Consistency

from:

"Agreement among related Information Elements (q.v.) in the data."

to:

"Agreement among related Information Elements (q.v.) that are present in the data. Note that missing Information Elements do not make a test Inconsistent."

ArthurChapman commented 2 years ago

I have tried to add COMPLETE and NOT_COMPLETE to the Glossary - wording needs checking @chicoreus

COMPLETE: An assertion of a MEASURE (q.v.) where the VALIDATION (_q.v.) Result.results (q.v.) from all included records in the dataset are COMPLIANT (q.v.).

NOT_COMPLETE: An assertion of a MEASURE (q.v.) where not all the VALIDATION (_q.v.) Result.results (q.v.) from all included records in the dataset are COMPLIANT (q.v.).

ArthurChapman commented 2 years ago

Looking at the definitions. We have a term

Test prerequisite - Prerequisites to the test being run in the form of: fields having values, tests that need to be run before the current test, availability of a specified source authority (q.v.), etc.

I am not sure we use this term anywhere anymore. We agreed that tests don't have an order (other than VALIDATION-AMENDMENT-VALIDATION. The other parts of the definition are handled by INTERNAL_PREREQUISITES_NOT_MET and EXTERNAL_PREREQUISITES_NOT_MET Do we just delete this term?