BDQ Core - VOCABULARY of terms

Tasilee commented 6 years ago

Terms in the bdqffdq namespace are from the Fitness for Use Framework (Viega et al. 2017). Use the reference to the Framework Definitions for more details and examples. The use of a vocabulary term in a test specification without a namespace prefix (sometimes represented in all UPPER CASE), implies that the bdq: or bdqffdq: namespace is applicable. Note that wherever "DQ" is used in a definition it implies "Data Quality" and wherever "FFU Framework" is used it refers to the "Fitness for Use Framework" (Veiga et al. 2017).

Note: There are two tables in this issue, the first is for vocabulary for the standard, the second is for additional terms for supplement files that will go into tables in those documents rather than controlled vocabularies.

Do not edit, moved to csv files

Pending further splits, this vocabulary moved to https://github.com/tdwg/bdq/blob/master/tg2/vocabularies/combined_vocabulary.csv

namespace:Term	Term	Definition	Context	Comment
bdqffdq:ActedUpon	ActedUpon	A subtype of an bdqffdq:InformationElement that is the primary focus of a test. InformationElements may also be Consulted, data quality reports should identify the ActedUpon terms of tests separately from Consulted terms.	bdqffdq:InformationElement
bdq:Alien-Species	Alien-Species	Research uses for occurrence data of alien species where 1) the information elements concern what organism occurred where and when and the means, degree, and pathways of establishment, and 2) that are used for analysis of spatial and/or temporal patterns of biodiversity (see examples in Groom et al. (2019). Improving Darwin Core for research and management of alien species. https://doi.org/10.3897/biss.3.38084).	bdqffdq:UseCase
bdq:AllAmendmentTestsRunOnSingleRecord	AllAmendmentTestsRunOnSingleRecord	A list of Amendments that have been run on a Single Record.	bdqffdq:InformationElements	Used in Measure of Single Record Tests
bdq:AllValidationTestsRunOnSingleRecord	AllValidationTestsRunOnSingleRecord	A list of Core Validation Tests that have been run on a Single Record.	bdqffdq:InformationElements	Used in Measure of Single Record Tests
bdq:Ambiguous	Ambiguous	Used to report where bdq:Conformance is not satisfied due to bdqffdq:InformationElements not being unambiguously resolvable by a bdq:sourceAuthority.	bdqffdq:DataQualityDimension
bdq:AMENDED	AMENDED	A bdq:Response.status used to indicate that a response for a bdqffdq:Amendment contains a proposed change to a record in the bdq:Response.result.	bdq:Response.status	Applies only to a bdqffdq:Amendment.
bdqffdq:Amendment	Amendment	A Data Quality needs level concept that describes a run of a test that proposes changes based on some data quality enhancement. The AMENDMENT concept in the Tests involves data that were amended by modification or addition of a value or values following defined bdqffdq:Criteria of a run result (bdq:Response) that includes a status of (bdq:AMENDED, bdq:FILLED_IN, bdq:TRANSPOSED, etc) as well as the proposed changes to values from the original data (in bdq:Response.value).	FFU Framework: Class	Formally in the Fitness for Use Framework (Veiga et al. 2017), the description of a test that can propose a change is a bdqffdq:Enhancement, while the corresponding report level concept is a bdqffdq:Amendment.
bdqffdq:AmendmentMethod	AmendmentMethod	A Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Enhancement in the context of bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:AmendmentPolicy	AmendmentPolicy	A Data Quality needs level concept that describes how some bdqffdq:contextualizedEnhancement relates to a bdqffdq:UseCase. This relationship defines which amendments are supported by a given use case.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:amendmentProperties	amendmentProperties	Sub properties of bdqffdq:ObjectProperties that apply to amendment concepts such as bdqffdq:AmendmentPolicy (DQ needs), bdqffdq:AmendmentMethod (DQ solutions) and bdqffdq:Amendment (DQ reports).	FFU Framework: ObjectProperty	Veiga et al. (2017).
bdqffdq:AmendmentReport	AmendmentReport	A Data Quality report level concept that results from a bdqffdq:Amendment that proposed changes to a bdqffdq:InformationElement.	FFU Framework: Class	Veiga et al. (2017).
bdq:annotationAlertIf	annotationAlertIf	Optionally establishes if an annotation exists within a bdq:annotationSystem by describing the criteria for relating annotations in the system to records in a bdq:ParameterizedTest.	bdq:Parameter	Used in test "ANNOTATION_ISSUE_NOTEMPTY" (fecaa8a3-bbd8-4c5a-a424-13c37c4bb7b1).
bdq:annotationSystem	annotationSystem	Optionally established a system for annotations within a bdq:ParameterizedTest with the default being the w3c Annotations Data Model's "oa:Annotation"	bdq:Parameter	Used in test "ANNOTATION_ISSUE_NOTEMPTY" (fecaa8a3-bbd8-4c5a-a424-13c37c4bb7b1).
bdqffdq:Assertion	Assertion	The bdqffdq:Assertion type in FFDQ is the fundamental concept that makes up a bdqffdq:DataQualityReport. bdqffdq:Assertion can be any one of four types (represented as subClasses), bdqffdq:Measure, bdqffdq:Validation, bdqffdq:Issue, and bdqffdq:Amendement.	FFU Framework: Class	Veiga et al. (2017). The assertion concept consists of a bdqffdq:Specification (the technical description of a performed test), a bdqffdq:DataResource (initial values of input data expressed in terms of some controlled vocabulary), the bdqffdq:Mechanism (external service, actor, or code that performs the test), and some form of bdqffdq:Result.
bdq:ASSUMEDDEFAULT	ASSUMEDDEFAULT	A bdqffdq:Amendment that replaces a bdq:EMPTY term with a predefined default bdq:Parameter value.	bdqTestField:Term-Actions	Would be used only in an extension or in bdq:Response.comment, bdq:Response.status value for this case is bdq:AMENDED.
bdq:assumptionOnUnknownBiome	assumptionOnUnknownBiome	Used when a bdq:taxonomyIsMarine source authority is unable to assert the marine or non-marine status of a taxon, the biome (marine/nonmarine) to assume instead or noassumption.	bdq:Parameter	See VALIDATION_COORDINATES_TERRESTRIALMARINE (b9c184ce-a859-410c-9d12-71a338200380).
bdq:Biotic-Relationships	Biotic-Relationships	Research uses for relationships between organisms where 1) the information elements concern what organisms have a relationship and 2) that are used for analysis of the relationship of one organism to another (see examples in Poelen JH, Simons JD, Mungall CJ. (2014). Global Biotic Interactions: An open infrastructure to share and analyze species-interaction datasets. Ecological Informatics, 24, 148–159. https://doi.org/10.1016/j.ecoinf.2014.08.005)	bdqffdq:UseCase
bdq:COMPLETE	COMPLETE	A bdqffdq:Assertion of a bdqffdq:Measure where data are present and sufficiently comprehensive for use.	bdq:Response.result
bdq:Completeness	Completeness	The extent to which data are present and sufficiently comprehensive for use.	bdqffdq:DataQualityDimension	Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:Conformance	Conformance	Conforms to a format, syntax, data type, range, or standard of the bdqffdq:InformationElement.	bdqffdq:DataQualityDimension	Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:COMPLIANT	COMPLIANT	A bdq:ExpectedResponse of a bdqffdq:Validation where the data conforms to the test bdqffdq:Criterion.	bdq:Response.result	Applies only to bdqffdq:Validations.
bdqffdq:ComposedOf	ComposedOf	Describes the properties from a controlled vocabulary that compose a bdqffdq:InformationElement. For example, a bdqffdq:InformationElement may be bdqffdq:composedOf properties such as dwc:day, dwc:month and dwc:year.	FFU Framework: ObjectProperty	Veiga et al. (2017).
bdq:Consistency	Consistency	Agreement among related bdqffdq:InformationElements that are present in the data. Note that missing bdqffdq:InformationElements do not make a test bdq:Inconsistent.	bdqffdq:DataQualityDimension	Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:CONSISTENT	CONSISTENT	Identifies inconsistency among values between bdqffdq:InformationElements.	bdqTestField:Term-Actions
bdqffdq:Consulted	Consulted	A bdqffdq:InformationElement that was referenced in a test but was not the primary focus of the test.	bdqffdq:InformationElements
bdqffdq:ContextualizedCriterion	ContextualizedCriterion	Describes an instance of the criterion concept in terms of the associated bdqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:ContextualizedDimension	ContextualizedDimension	Describes an instance of the bdqffdq:Dimension concept in terms of the associated bndqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:ContextualizedEnhancement	ContextualizedEnhancement	Describes an instance of the bdqffdq:Enhancement concept in terms of the associated bdqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:ContextualizedIssue	ContextualizedIssue	Describes an instance of the bdqffdq:Issue concept in terms of the associated bdqffdq:InformationElements from some controlled vocabulary (fields actedUpon or consulted), and a bdqffdq:ResourceType of bdqffdq:SingleRecord or bdqffdq:MultiRecord.	FFU Framework: Class	Veiga et al. (2017).
bdq:CONVERTED	CONVERTED	A conversion has been proposed to values in the bdqffdq:InformationElements to conform with a targeted reference system.	bdqTestField:Term-Actions	See Test "AMENDMENT_COORDINATES_CONVERTED" (620749b9-7d9c-4890-97d2-be3d1cde6da8).
bdq:COORDINATES	COORDINATES	Represents the combination of the Darwin Core terms dwc:decimalLatitude and dwc:decimalLongitude.	bdqffdq:InformationElement
bdqffdq:coversUseCase	coversUseCase	Used by concepts in the Data Quality needs category to describe the relationship between DQ Policies (bdqffdq:ValidationPolicy, bdqffdq:AmendmentPolicy, bdqffdq:MeasurementPolicy) and an instance of the bdqffdq:UseCase covered by that policy.	FFU Framework: ObjectProperty	Veiga et al. (2017).
bdqffdq:Criterion	Criterion	Describes the criterion a bdqffdq:Validation test uses to determine compliance. For example, "The value of dwc:basisOfRecord of bdqffdq:SingleRecords must be in the controlled vocabulary".	FFU Framework: Class	Veiga et al (2017).
bdqffdq:criterionInContext	criterionInContext	Describes the relationship between a bdqffdq:Validation concept in the FFU Framework (needs, solutions, reports) and a bdqffdq:contextualizedCriterion.	FFU Framework: ObjectProperty	Veiga et al. (2017).
bdq:dataID	dataID	The local (to bdq:ValidationData) integer indentifier for the Validation Data record	bdq:ValidationData
bdqffdq:DataQualityDimension	DataQualityDimension	Describes the aspect of data quality (accuracy, precision, completeness, etc.) that a test examines. For example, "precision" in "coordinate precision of single records". Includes Completeness (q.v.), Conformance (q.v.), Consistency (q.v.), Likeliness (q.v.), Reliability (q.v.), and Resolution (q.v.).	FFU Framework: Class	Note that the fail (bdq:NOT_COMPLIANT) assertions from running a test are one of: bdq:Ambiguous, bdq:Incomplete, bdq:Inconsistent, bdq:Invalid, or bdq:Unlikely.
bdqffdq:DataQualityReport	DataQualityReport	A set of bdqffdq:Assertions (bdqffdq:Measures, bdqffdq:Validations bdqffdq:Issues and bdqffdq:Amendments) that represent the output of a test run produced by a bdqffdq:Mechanism designed to assess the fitness for use of the tested data for a particular purpose as.	FFU Framework: Class	Veiga et al. (2017).	Fitness For Use Framework
bdqffdq:DataResource	DataResource	Describes a data resource described in terms of a controlled vocabulary such as dwc and represents the original values of the data operated on by an assertion test (i.e. an instance of dwc:Occurrence).	FFU Framework: Class	Veiga et al (2017).
bdq:DefaultSourceAuthority	DefaultSourceAuthority	A default where a required bdq:Parameter or a bdq:sourceAuthority (q.v.) has not been provided.	bdq:Parameter
bdq:defaultGeodeticDatum	defaultGeodeticDatum	Optionally established the default datum in a bdq:ParameterizedTest. A default datum is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter	See test AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT (7498ca76-c4d4-42e2-8103-acacccbdffa7).
bdq:defaultValue	defaultValue	A preselected value (e.g. year, elevation) where a required bdq:Parameter value has not been provided.	bdq:Parameter
bdqffdq:dimensionInContext	dimensionInContext	Describes the relationship between a bdqffdq:Amendment concept in the FFU Framework (needs, solutions, reports) and a bdqffdq:ContextualizedDimension.	FFU Framework: ObjectProperty	Veiga et al. (2017).
dwc:	dwc:	A namespace to indicate Darwin Core terms and which are listed in the dwcffdq:InformationElements of each Test.	Data
bdq:earliestValidDate	earliestValidDate	Optionally establishes the earliest date in a bdq:ParameterizedTest. A default date is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter
bdq:EMPTY	EMPTY	A bdqffdq:InformationElement that is either not present or does not contain any characters or values other than those in the range U+0000 to U+0020.	Data	Note: A bdqffdq:InformationElement containing invalid characters (e.g. letters in an information element that would be expected to contain integers) or values (including string serializations of the NULL value) are NOT_EMPTY and may be separately detected.
bdqffdq:Enhancement	Enhancement	Describes the enhancement to the original data performed by a bdqffdq:Amendment test. For example, "Recommends valid value for taxon name in a single record".	FFU Framework: Class	Veiga et al (2017).
bdqffdq:enhancementInContext	enhancementInContext	Describes the relationship between a bdqffdq:Amendment concept in FFU Framweork (needs, solutions, reports) and a bdqffdq:ContextualizedEnhancement.	FFU Framework: Object Property	Veiga et al. (2017)
EPSG:	EPSG	A pseudo-namespace referenced in dwc:datum to indicate the EPSG API where the numeric value following the colon is used as the search key. Example: EPSG:4326.	Data
bdq:Examples	Examples	Provide one pass (i.e. COMPLIANT) example and one fail (NON_COMPLIANT) example for each test.	bdq:Parameter
bdq:ExpectedResponse	ExpectedResponse	bdq:ExpectedReponse is one of the properties of a bdqffdq:Specification used in the markdown of the tests in the bdq GitHub.	bdq:Specification
bdq:EXTERNAL_PREREQUISITES_NOT_MET	EXTERNAL_PREREQUISITES_NOT_MET	A bdq;Response.status indicating that a bdq:Response.value was not generated because a bdq:sourceAuthority was not available or was off-line. If the test is run at a later time, it may produce a different result.	bdq:Response.status
bdq:FILLED_IN	FILLED_IN	A Response.status for a bdqffdq:Amendment where a value has been proposed for a bdqffdq:InformationElement that has no value.	bdq:Response.status
bdq:FOUND	FOUND	The value in a bdqffdq:InformationElement that matched a value in a bdq:sourceAuthority.	bdqTestField:Term-Actions	Use bdq:COMPLIANT for bdq:Response.result, and include this in bdq:Response.comments or bdq:Response.qualifier.
gbif:	gbif:	A pseudo-namespace referenced in dwc:taxonID to indicate the GBIF API where the numeric value following the colon is used as the search key. Example: gbif:8102122.	Data
bdq:GEOGRAPHY	GEOGRAPHY	A combination of Darwin Core administrative geography terms dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality.	bdqffdq:InformationElement
bdq:geospatialLand	geospatialLand	Polygons derived from a union of Natural Earth vectors for Land and for Minor Islands at 1:10,000,000 resolution.	bdq:Parameter	See VALIDATION_COORDINATES_TERRESTRIALMARINE (b9c184ce-a859-410c-9d12-71a338200380)
bdq:GUID	GUID	Gobally Unique Identifier. In this document, the GUID for a test is a UUID (128-bit universally unique identifier) which identifies the test.	Data	GUID is intended to identify the tests for machine consumption, "Label" is used for human consumption. https://en.wikipedia.org/wiki/Universally_unique_identifier
bdqffdq:hasCriterion	hasCriterion	Used to link the derived concept of a bdqffdq:ContextualizedCriterion to the fundamental concept of a bdqffdq:Criterion.	FFU Framework: ObjectProperty	Veiga et al (2017).
bdqffdq:hasDimension	hasDimension	Used to link the derived concept of a bdqffdq:ContextualizedDimension to the fundamental concept of a bdqffdq:Dimension.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:hasEnhancement	hasEnhancement	Used to link the derived concept of a bdqffdq:ContextualizedEnhancement to the fundamental concept of a bdqffdq:Enhancement.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:hasInformationElement	hasInformationElement	Provides a relationship between FFDQ concepts and the information elements. For example, bdqffdq:ContextualizedCriterion uses this property along with bdqffdq:hasResourceType to define a criterion in the context of related information elements.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:hasIssue	hasIssue	Used to link the derived concept of a bdqffdq:ContextualizedIssue to the fundamental concept of a Problem.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:hasResourceType	hasResourceType	Provides additional metadata, along with the bdqffdq:InformationElements, that describes the level (bdqffdq:SingleRecord or bdqffdq:MultiRecord) at which the FFDQ concept operates. For example, a bdqffdq:enhancementInContext with resource type of bdqffdq:MultiRecord could be used to define a bdqffdq:Amendment that applies at the level of multiple record values.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:hasSpecification	hasSpecification	Describes the relationship between a derived FFDQ concept and the fundamental concept of a bdqffdq:Specification (technical description of a test).	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:hasStatus	hasStatus	Used in the bdqffdqReport concept to describe result status. For example, in the case of a bdqffdq:Validation result, values could be bdq:COMPLIANT or bdq:NON_COMPLIANT.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:Implementation	Implementation	The FFDQ derived concept of a bdqffdq:Implementation describes the relationship between a bdqffdq:Specification (technical description of a test) and the bdqffdq:Mechanism that implements it.	FFU Framework: Class	Veiga et al (2017).
bdqffdq:implementedBy	implementedBy	Describes the link between the bdqffdq:Implementation concept in FFDQ and the bdqffdq:Mechanism that implements some bdqffdq:Specification (also defined in bdqffdq:Implementation).	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:ImprovementTarget	ImprovementTarget	The bdqffdq:ImprovementTarget concept in FFDQ describes which bdqffdq:Measures and bdqqffdq:Validations are improved by some bdqffdq:Amendment. bdqffdq:ImprovementTarget includes relationships between a bdqffdq:contextualizedEnhancement (for a bdqffdq:Amendment) and one or more bdqffdq:contextualizedCriterion (link to bdqffdq:Validations) or bdqffdq:contextualizedDimension (link to bdqffdq:Measures).	FFU Framework: Class	Veiga et al (2017).
bdqffdq:improvedBy	improvedBy	Object property that describes a bdqffdq:Enhancement, as part of the bdqffdq:ImprovementTarget, that would improve data acted upon by some set of bdqffdq:Measures or bdqffdq:Validations.	FFU Framework: Object Property	Veiga et al. (2017).
bdq:includeEventDate	bdq:includeEventDate	Allows dwc:eventDate to be excluded in a bdq:ParameterizedTest. The default is to include the event date in the test, but it may be excluded to allow an identification to be prior to the event date.	bdq:Parameter	Used in test "VALIDATION_DATEIDENTIFIED_INRANGE" (dc8aae4b-134f-4d75-8a71-c4186239178e).
bdq:Incomplete	Incomplete	Where a bdqffdq:InformationElement does not contain sufficient information to satisfy the scope of the test.	bdqffdq:DataQualityDimension
bdq:Inconsistent	Inconsistent	Where the Data Quality Dimension (q.v.): Consistency (q.v.) is not satisfied due to inconsistent values between the different Information Elements (q.v.) of a single record.	bdqffdq:DataQualityDimension
bdqffdq:InformationElement	InformationElement	A bdqffdq:InformationElement identifies a portion of data to which a test pertains. The bdqffdq:InformationElement in FFDQ can be represented as a single or composite element that consists of one or more terms from a controlled vocabulary (fields actedUpon or consulted by an assertion test) that identifies concepts in data relevant to a use case. An abstraction or a concrete term that represents relevant content (e.g., coordinates; dwc.decimalLatitude, dwc:decimalLongitude).	FFU Framework: Class	For the test descriptions, bdqffdq:InformationElements are concrete Darwin Core terms, to remove ambiguity for implementors. Veiga et al (2017).
bdq:INTERNAL_PREREQUISITES_NOT_MET	INTERNAL_PREREQUISITES_NOT_MET	A bdq:Response.status where values of the bdqffdq:InformationElement were insufficient to run the test. If the test is run at a later time on unmodified data, it should produce the same bdq:Response.	bdq:Response.status
bdq:interpretedAs	interpretedAs	(1) For Implementors, where Darwin Core data are serialized as strings, but the test refers to data as numeric or other non-string data type, can the string value be parsed into the target data type in the language of implementation (e.g., "1" as the integer 1), (2) matching a representation of a value unambiguously onto a controlled vocabulary (e.g., ‘WGS84’ to ’EPSG:4326’), or (3) interpreting the representation of a numeric value (e.g., a roman numeral) as a number (e.g., an integer).	Data
bdq:Invalid	Invalid	Where the bdqffdq:DataQualityDimension: bdq:Conformance is not satisfied due to bdqffdq:InformationElements containing non-standard values.	bdqffdq:DataQualityDimension
bdqffdq:Issue	Issue	A Data Quality needs level concept that flags issue or problems with the data. In the context of the tests, bdqffdq:Issue(s) are all either bdq:POTENTIAL_ISSUE, bdq:IS_ISSUE where potential problems are flagged and may need examination by the user to determine if data have quality for their use; or bdq:NOT_ISSUE.	FFU Framework: Class	Veiga et al (2017).
bdqffdq:issueInContext	issueInContext	Describes the relationship between a bdqffdq:Issue concept in FFU Framework (needs, solutions, reports) and a bdqffdq:ContextualizedIssue.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:IssueMethod	IssueMethod	A Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Issue in the context of bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements.	FFU Framework: Class	In Veiga et al. (2017) this was treated as "ProblemMethod"
bdqffdq:IssuePolicy	IssuePolicy	A Data Quality needs level concept that describes how some bdqffdq:contextualizedIssue relates to a bdqffdq:UseCase. This relationship defines which bdqffdq:Issues are supported by a given bdqffdq:UseCase.	FFU Framework: Class	In Veiga et al. (2017) this was treated as "ProblemPolicy"
bdqffdq:issueProperties	issueProperties	Sub properties of bdqffdq:ObjectProperties that apply to bdqffdq:Issue concepts such as bdqffdq:IssuePolicy (DQ needs), bdqffdq:IssueMethod (DQ solutions) and bdqffdq:Issue (DQ reports).	FFU Framework: Object Property	In Veiga et al. (2017) treated as "ProblemProperties"
bdqffdq:IssuesReport	IssuesReport	A Data Quality report level concept that results from a bdqffdq:Issue that flagged a problem in a test as bdq:IS_ISSUE, bdq:POTENTIAL_ISSUE or bdq:NOT_ISSUE.	FFU Framework: Class	Veiga et al. (2017).
bdq:IS_ISSUE	IS_ISSUE	A bdq:Response.result for a bdqffdq:Issue that flags where the data do not have sufficient quality for a use.	bdq:Response.result
rdfs:label	label	"See: https://www.w3.org/TR/rdf-schema/#ch_label"	RDF representation	"skos:prefLabel/skos:label may be prefered."
bdq:latestValidDate	latestValidDate	Optionally establishes the latest date in a bdq:ParameterizedTest. A default date is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter
bdq:Likeliness	Likeliness	The likelihood of Darwin Core Term(s) having true or expected values.	bdqffdq:DataQualityDimension	Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:LineNumber	LineNumber	The sequence number of the data record in the bdq:ValidationData	bdq:ValidationData
bdq:LineForTest	LineForTest	A local to bdq:ValidationData identifier for test records within one test	bdq:ValidationData
bdq:maximumValidDepthInMeters	maximumValidDepthInMeters	Optionally establishes the maximum depth in a bdq:ParameterizedTest. A default depth is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter
bdq:maximumValidElevationInMeters	maximumValidElevationInMeters	Optionally establishes the highest elevation in a bdq:ParameterizedTest. A default elevation is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter
bdqffdq:Measure	Measure	A Data Quality needs level concept that describes a run of a test that performs a measurement according to some data quality dimension. In FFDQ, the Measure concept consists of a run result of COMPLETE or NOT_COMPLETE, a value of the measurement (i.e. a measure of dwc:eventDate duration in seconds) or counts of the number of tests from a run where bdq:Response.result was bdq:NOT_COMPLIANT, or bdq:PREREQUISITES_NOT_MET in bdqffdq:Validation tests, or was bdq:AMENDED in bdqffdq:Amendment tests.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:MeasurementMethod	MeasurementMethod	A Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Dimension in the context of bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements.	FFU Framework: Class	Veiga et al (2017).
bdqffdq:MeasurementPolicy	MeasurementPolicy	A Data Quality needs level concept that describes how some bdqffdq:contextualizedDimension relates to a bdqffdq:UseCase. This relationship defines which bdqffdq:Measures are supported by a given bdqffdq:UseCase.	FFU Framework: Class	Veiga et al (2017).
bdqffdq:measurementProperties	measurementProperties	Sub properties of bdqffdq:ObjectProperties that apply to measurement concepts such as bdqffdq:MeasurementPolicy (DQ needs), bdqffdq:MeasurementMethod (DQ solutions) and bdqffdq:Measure (DQ reports).	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:MeasurementReport	MeasurementReport	A Data Quality report level concept that describes the results of a run of a test that performs a measurement according to some data quality dimension.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:Mechanism	Mechanism	The FFDQ concept of bdqffdq:Mechanism describes the entity that performs an assertion test (code, external service, actor, etc.). Tied to a bdqffdq:Specification via the concept of a bdqffdq:Implementation.	FFU Framework: Class	Veiga et al (2017).
bdq:minimumValidDepthInMeters	minimumValidDepthInMeters	Optionally establishes the minimum depth in a bdq:ParameterizedTest. A default depth is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter
bdq:minimumValidElevationInMeters	minimumValidElevationInMeters	Optionally establishes the lowest elevation in a bdq:ParameterizedTest. A default elevation is supplied in cases where a bdq:Parameter is not set at the time the test is run.	bdq:Parameter
bdqffdq:MultiRecord	MultiRecord	A data set composed of one or more bdqffdq:SingleRecords.	FFU Framework: namedIndividual	Veiga et al. (2017).
non-printing characters	non-printing characters	ASCII 0-32 and 127 decimal. Non printing characters or formatting marks that are not displayed at printing. These may include pilcrow, space, non-breaking space, tab character. etc. For the purposes of the tests they are treated as bdq:EMPTY.	Data
bdq:NOT_AMENDED	NOT_AMENDED	A bdq:Result.status that indicates that a bdq:Response for a bdqffdq:Amendment proposed no change.	bdq:Response.status
bdq:NOT_COMPLETE	NOT_COMPLETE	An assertion of a bdqffdq:Measure on a bdqffdq:MultiRecord where not all the bdqffdq:Validation bdq:Response.result from all included records in the dataset are bdq:COMPLIANT.	bdq:Response.result	The scope in the Fitness of Use Framework (Veiga et al. (20117) is broader.
bdq:NOT_COMPLIANT	NOT_COMPLIANT	A bdq:Response.result of a bdqffdq:Validation where the data do not conform to the Test bdqffdq:Criterion.	bdq:Response.status
bdq:NOT_ISSUE	NOT_ISSUE	The bdq:Response of a test of type bdqffdq:Issue where no potential problems were detected.	bdq:Response.result
bdq:NOTEMPTY	NOTEMPTY	The value of a bdqffdq:InformationElement that is present and has content (cf. bdq:EMPTY)	Data
null	null	A value that is used in some databases to signify that a value is unknown or missing. It may be represented in serializations by "NULL", "Null", "null". "/n", "9999", etc. These should be treated as bdq:NOTEMPTY.	Data
bdq:OUTOFRANGE	OUTOFRANGE	The value in a bdqffdq:InformationElement that is outside an acceptable range for that bdqffdq:InformationElement.	bdqTestField:Term-Actions	Use in bdq:Response.qualifier or bdq:Response.comment.
bdq:Parameter	Parameter	A value provided to a test that changes the behavior of a test to fit a particular user need within the scope of the test. Either 1) a link to a bdq:sourceAuthority to find matching values, or 2) a value used to define limits for a bdqffdq:InformationElement.	Data
bdq:paramaterizedTest	paramaterizedTest	A test that allows a bdq:Parameter to be set prior to the test being run. Where a bdq:Parameter value has not been provided, a default is specified within the test.	Test
bdq:POLYNOMIAL	POLYNOMIAL	Represents a combination of the Darwin Core terms dwc:genericName, dwc:specificEpithet, dwc:infraspecificEpithet.	bdqffdq:InformationElement	See test "VALIDATION_POLYNOMIAL_CONSISTENT" (17f03f1f-f74d-40c0-8071-2927cfc9487b)
bdq:POTENTIAL_ISSUE	POTENTIAL_ISSUE	A bdq:Response.result for a bdqffdq:Issue that flags where the data may not have sufficient quality for a use. See also bdq:IS_ISSUE and bdq:NOT_ISSUE. The user will need to evaluate if the data is fit for their particular use or not.	bdq:Response.result
bdq:PRECISIONINSECONDS	PRECISIONINSECONDS	The length of the period of an event in seconds.	bdqTestField:Term-Actions	This is description of the bdq:Response.result from this bdqffdq:Measure, where the result is a numeric value in seconds. See Test "MEASURE_EVENTDATE_DURATIONINSECONDS" (56b6c695-adf1-418e-95d2-da04cad7be53).
skos:prefLabel	Preferred Label	See https://www.w3.org/TR/skos-reference/#labels.	SKOS Representation	from SKOS Simple Knowledge Organization System
bdq:PREREQUISITESNOTMET	PREREQUISITESNOTMET	A test of type bdqffdq:Measure that counts the number of tests of type bdqffdq:Validation that did not run due to one or more prerequisites not being met (e.g. bdq:INTERNAL_PREREQUISITES_NOTMET and bdq:EXTERNAL_PREREQUISITES_NOTMET)	bdqTestField:Term-Actions	See test "MEASURE_VALIDATIONTESTS_PREREQUISITESNOTMET" (49a94636-a562-4e6b-803c-665c80628a3d).
bdqffdq:Profile	Profile	a Data Quality needs level concept describing the bdqffdq:UseCases that make up some data quality operation such as the behavior of a single actor or workflow producing the relevant bdqffdq:Assertions.	FFU Framework: Class	Veiga et al. (2017).
bdq:PROPOSED	PROPOSED	A test of type bdqffdq:Measure that pertains to a bdqffdq:Amendment where an action to modify a value in some way through a change or addition is recommended.	bdqTestField:Term-Actions	Example see test "MEASURE_AMENDMENTS_PROPOSED" (03049fe5-a575-404f-b564-ae63f5a1cf8b).
bdq:Record-Management	Record-Management	Management of the quality of biodiversity data records (see examples in Rees ER & Nicholls M (2020) Data Quality Use Case Study Results https://doi.org/10.3897/biss.4.50889.suppl2).	bdqffdq:UseCase
bdq:Reliability	Reliability	Measure of how the data values agree with an identified source of truth. The degree to which data correctly describes the truth (object, event or any abstract or real 'thing').	bdqffdq:DataQualityDimension	Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:Resolution	Resolution	Refers to the data having sufficiently detailed information. Measure of the granularity of the data, or the smallest measurable increment.	bdqffdq:DataQualityDimension	Definition from the Fitness for Use Framework: Data Quality Dimensions Document (Link needed to RDF document - https://github.com/tdwg/bdq/wiki/TG2-Data-Quality-Dimension).
bdq:Response	Response	The report from a single execution of a single test, consisting of bdq:Response.status, bdq:Response.result, bdq:Response.comment, and optionally, bdq:Response.qualifier.	bdq:Response	Parent of RESULT and RESULT_STATUS in the Fitness for Use Framework (Viega et al. 2017).
bdq:Response.comment	Response.comment	Human readable interpretation of the results of the test.	bdq:Response
bdq:Response.qualifier	Response.qualifier	Additional structured information that qualifies the bdq:Response, intended as an extension point for uncertainty.	bdq:Response
bdq:Response.result	Response.result	The element in a bdq:Response containing the value returned by the particular test (bdqffdq:Validation, bdqffdq:Amendment, bdqffdq:Measure, or bdqffdq:Issue)	bdq:Response
bdq:Response.status	Response.status	A metadata element in a bdq:Response indicating whether a particular test (bdqffdq:Validation, bdqffdq:Amendment, bdqffdq:Measure, or bdqffdq:Issue) was able to be performed or not.	bdq:Response
bdqffdq:ResourceType	ResourceType	In FFDQ the concept of bdqffdq:ResourceType has instances for bdqffdq:SingleRecord or bdqffdq:MultiRecord.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:Result	Result	The report bdqffdq:Result concept in FFDQ is represented as a value or a bdqffdq:ResultStatus for bdqffdq:Measures, just a bdqffdq:ResultStatus for bdqffdq:Validations and a bdqffdq:ResultStatus as well as values for changes proposed by bdqffdq:Amendments.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:ResultStatus	ResultStatus	Depending on the assertion type would have values of bdq:COMPLIANT or bdq:NOT_COMPLIANT for a bdqffdq:Validation, bdq:COMPLETE or bdq:NOT_COMPLETE for a bdqffdq:Measure, bdq:AMENDED, bdq:FILLED_IN, bdq:TRANSPOSED, bdq:NOT_AMENDED for a bdqffdq:Amendment and bdq:IS_ISSUE, bdq:POTENTIAL_ISSUE or bdq:NOT_ISSUE for a bdqffdq:Issue	FFU Framework: Class	Veiga et al. (2017). Note that a separate concept describes the resultstate with values of bdq:INTERNAL_PREREQUISITES_NOT_MET and bdq:EXTERNAL_PREREQUISITES_NOT_MET.
Roman numerals	Roman numerals	Roman numerals are interpreted as the equivalent integer for months (e.g. "X" as "10") in appropriate tests. Roman numerals may not be unambiguously interpreted for other Darwin Core terms such as dwc:day or in text fields as they may mean unknown or something else entirely.	Data
bdq:RUN_HAS_RESULT	RUN_HAS_RESULT	A bdq:Response.status that implies that a result was correctly generated.	bdq:Response.status	Applies to bdqffdq:Validations, bdqfdfq:Measures, and bdqffdq:Issues, but not bdqffdq:Amendments. See Fitness for Use Framework definition in Need link to OWL Document @chicoreus. See also bdq:INTERNAL_PREREQUISITES_NOT_MET and bdq:EXTERNAL_PREREQUISITES_NOT_MET
bdqffdq:SingleRecord	SingleRecord	A record from a dataset without dependencies on any other record.	FFU Framework: namedIndividual	Veiga et al. (2017). Note that all the current tests are run on a bdqffdq:SingleRecord, of Darwin Core data, and not designed to be run across a bdqffdq:MultiRecord, except for bdqffdq:MultiRecord bdqffdq:Measures.
bdq:sourceAuthority	sourceAuthority	An authority using the "bdq" namespace that provides a reference for values required for a test evaluation. Where the test is a bdq:ParameterizedTest a bdq:defaultSourceAuthority ("bdq:sourceAuthority default = xxx") is specified.	bdq:Parameter
bdq:spatialBufferInMeters	spatialBufferInMeters	A buffer in meters from a polygon (geopolitical boundary, coastline, etc.).	bdq:Parameter
bdq:Spatial-Temporal Patterns	Spatial-Temporal Patterns	Research uses for biodiversity occurrence data where 1) the information elements concern what organism occurred where and when and 2) that are used for analysis of spatial and/or temporal patterns of biodiversity (see examples in Rees ER & Nicholls M (2020) Data Quality Use Case Study Results https://doi.org/10.3897/biss.4.50889.suppl2).	bdqffdq:UseCase
bdqffdq:Specification	Specification	A technical description of the performed test upon which a bdqffdq:Implementation could be made.	FFU Framework: Class
bdq:STANDARD	STANDARD	A bdqffdq:Amendment where a value in a bdqffdq:InformationElement is proposed from a bdq:sourceAuthority.	bdqTestField:Term-Actions	Use in bdq:Response.qualifier or bdq:Response.comment.
bdq:STANDARDIZED	STANDARDIZED	A bdqffdq:Amendment where a bdq:STANDARD value for a bdqffdq:InformationElement is proposed.	bdqTestField:Term-Actions	Use bdq:AMENDED as the bdq:Response.status, report bdq:STANDARDIZED in a bdq:Response.qualifier or in a bdq:Response.comment.
bdq:targetCRS	targetCRS	The Coordinate Reference System (CRS) used as the output when converting coordinates from one CRS to another. The default is EPSG:4326.	bdq:Parameter	Used in the test AMENDMENT_COORDINATES_CONVERTED (620749b9-7d9c-4890-97d2-be3d1cde6da8).
bdqffdq:targetedCriterion	targetedCriterion	The bdffdq:Criterion targeted by some enhancement via the bdqffdq:ImprovementTarget object.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:targetedDimension	targetedDimension	The bdqffdq:Dimension targeted by some enhancement via the bdqffdq:ImprovementTarget object.	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:targetedIssue	targetedIssue	The bdqffdq:Issue targeted by some problem via the bdqffdq:ImprovementTarget object.	FFU Framework: Object Property	Veiga et al. (2017).
bdq:Taxon-Management	Taxon-Management	Management of the quality of taxonomic names (see examples in Rees ER & Nicholls M (2020) Data Quality Use Case Study Results https://doi.org/10.3897/biss.4.50889.suppl2).	bdqffdq:UseCase
bdq:taxonIsMarine	taxonIsMarine	Marine/non-marine status obtained from the World Register of Marine Species (WORMS) database	bdq:Parameter	See VALIDATION_COORDINATES_TERRESTRIALMARINE (b9c184ce-a859-410c-9d12-71a338200380).
bdq:TERRESTRIALMARINE	TERRESTRIALMARINE	A terrestrial taxon that has geographic coordinates that fall within terrestrial boundaries; or a marine taxon that has geographic coordinates that fall within marine boundaries.	bdqTestField:Term-Actions	Use bdq:AMENDED as the bdq:Response.status, report bdq:TERRESTRIALMARINE in a bdq:Response.qualifier or in a bdq:Response.comment. See test "VALIDATION_COORDINATES_TERRESTRIALMARINE" (b9c184ce-a859-410c-9d12-71a338200380).
bdq:TestField	testFields	Column heading in the markdown of the tests in the bdq GitHub that list all the normative and informative metadata elements that describe a Data Quality Test	Test
bdq:TestPrerequisite	TestPrerequisite	Conditions that must be met for a test to be run (e.g., fields having values, tests that need to be run before the current test, availability of a bdq:sourceAuthority)	Test	See for example, INTERNAL_PREREQUISTES_NOT_MET (q.v.) and EXTERNAL_PREREQUISITES_NOT_MET (q.v.).
bdq:TestType	TestType	There are four types of tests, viz. bdqffdq:Validation, bdqffdq:Amendment, bdqffdq:Issue, and bdqffdq:Measure.	Test
bdq:TRANSPOSED	TRANSPOSED	The sign and/or value of one or more bdqffdq:InformationElements were swapped.	bdqTestField:Term-Actions	Use bdq:AMENDED as the bdq:Response.status, report bdq:TRANSPOSED in a bdq:Response.qualifier or in a bdq:Response.comment. See Test "AMENDMENT_COORDINATES_TRANSPOSED" (f2b4a50a-6b2f-4930-b9df-da87b6a21082).
bdq:Unlikely	Unlikely	The bdqffdq:DataQualityDimension: bdq:Likeliness is not satisfied due to the bdq:InformationElements containing a value that is not likely to occur (for example where the geographic coordinates are "0", "0").	bdqffdq:DataQualityDimension	Needs a bdq:Response.qualifier (q.v.) response for the uncertainty.
bdqffdq:UseCase	UseCase	The bdqffdq:UseCase concept in FFDQ describes some data quality control use case. The bdqffdq:Amendment, bdqffdq:Measure and bdqffdq:Validation policies that make up a bdqffdq:UseCase define which bdqffdq:Assertions cover a given bdqffdq:UseCase.	FFU Framework: Class	An example of a bdqffdq:UseCase could be "Check for internal consistency of dates", with bdqffdq:Validation policies for checking consistency between atomic date fields and a bdqffdq:Amendment such as "eventDate filled in from verbatim". Veiga et al. (2017).
bdqffdq:Validation	Validation	A Data Quality needs level concept that describes a run of a test for validity. The bdqffdq:Validation concept in the Tests consists of a run with a bdq:Response:result of bdq:COMPLIANT or bdq:NOT_COMPLIANT and a bdqffdq:Criterion that describes the conditions for validity that result in a status of bdq:COMPLIANT.	FFU Framework: Class	Veiga et al. (2017).
bdq:ValidationData	ValidationData	Test data set established for testing Test Implementations	Data
bdqffdq:ValidationMethod	ValidationMethod	TA Data Quality solutions level concept describing the relationship between a bdqffdq:Specification (technical description of a test) and a bdqffdq:Criterion in the context of a bdqffdq:ResourceType (bdqffdq:SingleRecord or bdqffdq:MultiRecord) and associated bdqffdq:InformationElements.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:ValidationPolicy	ValidationPolicy	A Data Quality needs level concept that describes how some bdqffdq:contextualizedCriterion relates to a bdqffdq:UseCase. This relationship defines which validations are supported by a given bdqffdq:UseCase.	FFU Framework: Class	Veiga et al. (2017).
bdqffdq:validationProperties	validationProperties	Sub properties of bdqffdq:ObjectProperties that apply to validation concepts such as bdqffdq:ValidationPolicy (DQ needs), bdqfdq:ValidationMethod (DQ solutions) and bdqffdq:Validation (DQ reports).	FFU Framework: Object Property	Veiga et al. (2017).
bdqffdq:ValidationReport	ValidationReport	A Data Quality report level concept that reports the results of a run of a bdqffdq:Validation test on some data.	FFU Framework: Class	Veiga et al. (2017).
bdq:VERBATIM	VERBATIM	An original value.	bdqffdq:InformationElement
white space	white space	1) A field that only includes white space (blanks) is treated as bdq:EMPTY (q.v.). 2) In bdqffdq:Validation tests (q.v.) that require the looking up of a bdq:sourceAuthority, leading and/or trailing white space will cause the test to fail as no preprocessing is carried out on the data. These leading and trailing white spaces may be stripped out in a subsequent bdqffdq:Amendment and thus pass when the bdqffdq:Validation test is run again.	Data
bdq:YEARMONTHDAY	YEARMONTHDAY	Represents a combination of the Darwin Core terms dwc:year, dwc:month, dwc:day.	bdqffdq:InformationElement
bdq:YEARSTARTDAYOFYEARENDDAYOFYEAR	YEARSTARTDAYOFYEARENDDAYOFYEAR	Represents a combination of the Darwin Core terms dwc:year, dwc:startDayOfYear, dwc:endDayofYear.	bdqffdq:InformationElement

Supplement: GitHub Label Terms These are terms that are outside the Standard but that have been used as either GitHub Labels or TestFields in the BDQ GitHub

Do not edit, moved to csv files

Pending further moves, this vocabulary moved to https://github.com/tdwg/bdq/blob/master/tg2/vocabularies/glossary_terms.csv

namespace:Term	Label	Definition	Context	Comment
bdqtag:Amendment	Amendment	A label to indicate a test of type AMENDMENT which may propose a change or addition to at least one Darwin Core term that is intended to improve one or more components of the quality of the record.	GitHub Label	See bdqffdq:Amendment
bdqtag:CORE	CORE	Tests for evaluating biodiversity data quality as represented by the values of Darwin Core terms. CORE tests address identified user needs, are widely applicable, informative, unambiguous, well defined, and straight forward to implement.	GitHub Label
bdqTestField:Darwin Core Class	Darwin Core Class	The Information Element in the original terms of the framework, the general sort of information this test operates on.	TestField
bdqTestField:Data Quality Dimension	Data Quality Dimension	The data quality dimension for this test. See bbqffdq:DataQualityDimension.	TestField
bdqTestField:Description	Description	A non-technical description of what the test does, intended for consumers of data quality reports in concert with the bdq:Response.comment.	TestField
bdqtag:DO NOT IMPLEMENT	DO NOT IMPLEMENT	Tests that are not CORE (cf. bdqtag:CORE) and not recommended to be implemented with the current level of understanding for one or more reasons: Available vocabularies are ambiguous; the test is too complex to implement concisely; implementation is expected to lead to ambiguous or inaccurate results.	GitHub Label
bdqTestField:Example Implementations (Mechanisms)	Example Implementations (Mechanisms)	Known Mechanisms with implementations of the test.	TestField
bdqTestField:Examples	Examples	A ’pass’ and a ‘fail’ example of test data. All examples listed are present in the the validation data suite.	TestField
bdqTestField:Expected Response	Expected Response	The specification for implementors describing the expected behavior of the test. See bdqffdq:Specification	TestField	= bdqffdq:Specification
bdqTestField:GUID	GUID	see bdq:GUID	TestField
bdqtag:Immature/Incomplete	Immature/Incomplete	Tests where substantial work is needed to develop the specification to the point where the test can be reliably and usefully implemented. This may indicate work that is wholly internal to the test specification such as developing a consistent Expected Response, or may indicate that external work is needed to develop an agreed vocabulary for values of the tested term. An immature/incomplete test may be made CORE, Supplementary, or DO NOT IMPLEMENT when relevant criteria are satisfied.
bdqTestField:Information Elements Acted Upon	Information Elements Acted Upon	A list of the specific Darwin Core terms that are the focus of a test.	TestField
bdqTestField:Information Elements Consulted	Information Elements Consulted	AA list of Darwin Core terms that are consulted in the evaluation of the Information Elements ActedUpon.	TestField
bdqtag:ISO/DCMI STANDARD	ISO/DCMI STANDARD	A reference to either an ISO (International Organization for Standardization) Standard or a DCMI (Dublin Core Metadata Initiative) Standard	GitHub Label
bdqtag:Issue	Issue	A label to indicate a test of type ISSUE, where potential problems are flagged and may need examination by the user to determine if data have quality for their use.	GitHub Label	see bdqffdq:Issue
bdqTestField:Label	Label	A human readable label identifying the test. The labels largely follow the pattern TYPE_INFORMATIONELEMENT_STATUS.	TestField	cf. rdfs:label
bdqTestField:Link to Specification Source Code	Link to Specification Source Code	A link to code that implements the test.	TestField
bdqtag:Measure	Measure	A label to indicate a test of type MEASURE that performs a measurement according to some data quality dimension.	GitHub Label	See bdqffdq:Measure
bdqtag:NAME	NAME	A label to indicate that the test is related to Darwin Core terms in the dwc:Taxon Class.	GitHub Label
bdqtag:NEEDS WORK	NEEDS WORK	A label that indicates that an issue (Test) requires more work before finalising.	GitHub Label
bdqTestField:Notes	Notes	Additional, non-normative comments that the Task Group believed necessary for an accurate understanding of the test or issues that implementers needed to be aware of.	TestField
bdqtag:OTHER	OTHER	A label to indicate that the test is related to Darwin Core terms other than Classes dwc:Taxon, dwc:Location or dwc:Event.	GitHub Label
bdqtag:Parameterized	Parameterized	A label for a test that requires a bdq:Parameter to be set prior to a bdq:parameterizedTest being run.	GitHub Label
bdqTestField:Parameter(s)	Parameter(s)	Any parameters that change the behavior of the test for a subset of users with special data quality needs within the domain.	TestField
bdqTestField:References	References	A list of references pertinent to the test.	TestField
bdqTestField:Source	Source	The origin of the concept of the test.	TestField
bdqTestField:Source Authority	Source Authority	A reference to an external (non-Darwin Core) authority required for the test. See bdq:sourceAuthority	TestField
bdqtag:SPACE	SPACE	A label to indicate that the test is related to Darwin Core terms in the dwc:Location Class.	GitHub Label
bdqTestField:Specification Last Updated	Specification Last Updated	The last date a change was made to a test that affects the operation of the test.	TestField
bdqtag:Supplementary	Supplementary	Tests regarded as not CORE (cf. bdqtag:CORE) because of one or more reasons: Not widely applicable; not clearly matched to an identified data quality need; not informative concerning the 'quality' or lack of quality of the data; likely to return a high percentage of either bdq:NOT_COMPLIANT or bdq:POTENTIAL_ISSUE records. A Supplementary test MAY be implemented in a local implementation if a suitable use case exists.	GitHub Label	A Supplementary test may be made CORE at a later time.
bdqTestField:Term-Actions	Term-Actions	Equivalent to the bdqTestField:Label without the leading Test Type.	TestField
bdqtag:Test	Test	Tests descriptions created by TG2, either CORE, Immature/Incomplete, Supplementary, or DO NOT IMPLEMENT.	GitHub Label
bdqTestField:Test Type	Test Type	The Type of assertion that the test produces, Measure, Validation, Amendment, Issue.	TestField
bdqtag:TG1	TG1	Issues pertinent to Task Group 1 (Framework on Data Quality) of the TDWG Data Quality Interest Group.	GitHub Label
bdqtag:TG2	TG2	Issues including Tests, developed by, or pertinent to Task Group 2 (Data Quality Tests and Assertions) of the TDWG Data Quality Interest Group.	GitHub Label
bdqtag:TG3	TG3	Issues pertinent to Task Group 3 (Data Quality Use Cases) of the TDWG Data Quality Interest Group.	GitHub Label
bdqtag:TG4	TG4	Issues pertinent to Task Group 4 (Best Practices for Development of Vocabularies of Value) of the TDWG Data Quality Interest Group.	GitHub Label
bdqtag:TIME	TIME	A label to indicate that the test is related to Darwin Core terms in the dwc:Event Class.	GitHub Label
bdqtag:Validation	Validation	A label to indicate a test of type VALIDATION that describes a run of a test for validity against a set of criteria.	GitHub Label	See bdqffdq:Validation
bdqtag:VOCABULARY	VOCABULARY	A label to indicate that a bdqlabel:Test requires a Vocabulary	GitHub Label

ArthurChapman commented 6 months ago

I wouldn't have #TG3" in the definition something like

Tests of biodiversity data (represented by Darwin Core term values) that have been informed by use cases, that are widely applicable, informative in terms of fitness-for-use (or lack of it), and are readily implementable without ambiguity. Tests that are not considered CORE are one of: bdqtag:Supplementary, bdqtab:NO NOT IMPLEMENT, or bdq:Immature/Incomplete.

chicoreus commented 6 months ago

@ArthurChapman That's clear. One typo, s/NO NOT IMPLEMENT/DO NOT IMPLEMENT. We should also include the word mature, as immature tests can meet the other criteria of this sense of CORE.

We will now need to be explicit about UseCases, sensu the framework, and assign each CORE test to at least one UseCase.

We've got one clear UseCase as "Research analysis of biodiversity occurrence data of which organisms have been reported to occur where at what dates" We need to be similarly explicit about a UseCase, probably covering checklist data, that incorporates the latest set of CORE tests about pathway, establishmentmeans, degreeofestablisment, and frame a use case that incorporates type status validation, probably a taxonomic data use case, which would then incorporate validation of recorded by (which I think we've currently got as supplementary).

chicoreus commented 6 months ago

But, the text for the CORE project here states that it is: "Links to Confirmed Tests and Assertions arising out of Task Group 2." If we expand CORE beyond this, we still have to define the new use cases, in detail, likely a several year task. Best route is to keep CORE to the research uses of what organisms occurred where when use case that came out of TG2, and specify an additional category of tests that aren't core. But if they are being put forward as part of the standard they will need to have at use cases to hang them off of.

tucotuco commented 6 months ago

I would like to propose an alternative approach as none of us can afford years more of working on this before proposing a standard. I want to repeat my plea for simplicity and a decoupling of tests from use cases.

I see the set of tests as parallel to the Darwin Core bag of terms and and the use cases as parallel to the distinct Darwin Core Archive cores in terms of combining Darwin Core terms for a particular purpose. I think the tests should not depend on use cases. It should be the other way around. I think a use case should be a level of construct (a profile we called it before) that brings together a set of tests on a declared set of Darwin Core terms and that can declare data quality measures based on their values.

With this approach we could make the one occurrence use case based on the TG3 work as a model to show how that is done and let future work define new use cases as demand arises. These could be the stuff of task groups, and would be more tractable the less monolithic we make the standard. This is how I have thought about the BDQ work since the beginning, and ever more so now that the tribulations of coupled tests and use cases is creating a seemingly insurmountable obstacle.

This would leave us free of the otherwise somewhat arbitrary and controversial distinctions of CORE, SUPPLEMENTARY and DO NOT IMPLEMENT. The ones we finalize for the use case would become part of the standard set of tests. The rest just remain documented in GitHub (with their labels, also documented), but nowhere in the standard documentation. Having all of the rest in the standard just seems like noise to me. Put in the standard that which is mature and useful for a given purpose and leave the rest as a solid basis for future work if demand arises.

ArthurChapman commented 6 months ago

I 100% agree with @tucotuco. TG3 was a proof of concept and was never meant to be a comprehensive set of tests. I, like @tucotuco "This is how I have thought about the BDQ work since the beginning, and ever more so now that the tribulations of coupled tests and use cases is creating a seemingly insurmountable obstacle." I also have, until recently, seen the tests types as "somewhat arbitrary and controversial distinctions of CORE, SUPPLEMENTARY and DO NOT IMPLEMENT. The ones we finalize for the use case would become part of the standard set of tests." Originally treating these as basically a 1) a good test we can implement practically and that will be useful (CORE) 2). A tests that for the reasons stated in the definition are not ready to be Core tests and we only include them in the GitHub to not waste the work we've already done - not for the Standard but but may be mentioned (as a group, not individually) in the document for documentation purposes (SUPPLEMENTARY) 3). Tests that are close to being CORE but need more work because something is missing - e.g. a suitable Vocabulary (Immature/Incomplete) - note a couple may become CORE before we release the Standard if suitable Vocabularies become available and 4). Tests that for some reason we believe should not be implemented as doing so could lead to ambiguous or misleading results (DO NOT IMPLEMENT).

I know that I, and others, are getting at the limit of my capacity to continue with this work and want to see it finished - so let's keep it simple. The suggestions of @tucotuco I support and would vote to continue in that direction - not getting bogged down on things that will not go in the Standard - including both non CORE tests, and detailed Use Cases for each test. We all know Use Cases exist for the tests but to fully document them all now would take another two years of work at least (103 tests @ conservatively 2 days work per test = 206 days work!).

chicoreus commented 6 months ago

The problem is that we cannot describe tests within the framework without attaching them to UseCases. Fittness for purpose is a fundamental of the framework, and all tests within the framework descend from a UseCase. As long as CORE matched up with the broad use case that came out of TG3, research uses of data describing what organisms occurred where when, we could ignore this, as all of the tests we were working on hung off of that use case. The moment we expanded to consider tests outside of that scope, we are forced to define additional use cases. For supplementary tests, we can probably get away with something very skeletal that later users of those tests would replace with more clearly specified use cases. But we can't get away with this for the set of tests now in CORE.

To use the framework, we cannot decouple the tests from use cases. We must provide at least one very clear example of how a set of tests links to a use case. This will be central to the normative RDF representation of the tests.

We can think of tests as a grab bag, and the framework enables people to assemble tests from that grab bag to fit a use case. But, central to the standard must be a demonstration of how to do that. We had that with CORE when it precicely overlapped the broad use case. We don't right now.

@tucotuco is probably pointing us in the right direction of a grab bag of tests, paralell to darwin core terms that can be assembled as needed by users for their use cases. But for this to work, and for us to be able to use the framework, we have to clearly specify at least one use case and link a set of tests to that use case to show how this is done for both quality assurance and quality control.

ymgan commented 6 months ago

It's interesting to see everyone's perspectives in this, I appreciate this discussion, thank you! I don't know enough of what TG3 did to comment on the use cases, but I would like to share my perspective when I was mapping the checks from OBIS with the tests here:

It was very useful for me when @Tasilee told us that the CORE tests should be the first sets of tests that we should look at. We felt reassured that tests labeled with CORE are well developed and should be mature enough for our work. To me (as a user) at least, this is more important:

Concept 2: CORE: The set of mature tests that TG2 is putting forward as part of the standard. This is the bit meant by "Darwin Core terms that are widely applicable, informative, and straight forward to implement"

At that time, there was no supplementary tests yet, so I can't comment on this.
Notes from DO NOT IMPLEMENT was extremely helpful - save us time so that we know that we do not need to consider those.
Adapting the parameters (source authority) of some of the CORE tests are sufficient for our marine biodiversity data use case (e.g. using WoRMS as taxonomic backbone for https://github.com/tdwg/bdq/issues/46) we did not need to look further.
All in all, I agreed with how @ArthurChapman look at the labels in this comment
I felt that having example use case could be helpful but eventually, which tests are useful for any use case will depend on what the use case is and what data (and in what form, verbatim or not) the user has in their database.

The use case that we did in the OBIS data quality project team:

We came with the question - Can we have a set of standardized tests that could help to evaluate the data quality for records with Darwin Core fields that are deemed important to marine biodiversity data (mostly the required terms and strongly recommended terms for OBIS)?
We treated the tests as a grab bag and mapped the checks of OBIS portal to the CORE tests last year: https://github.com/iobis/quality-taskteam/wiki/Mapping-of-checks-in-obis%E2%80%90qc-to-TDWG-BDQ-core-tests-and-assertions
Report of those checks: https://r.obis.org/quality/ although not all the tests have been updated with the parameters the team has decided. This is the work of OBIS Secretariat.
Data managers of various OBIS nodes are expected to update the data with corrections (amendments).

Another thing that I think MAY be helpful is to clarify what CORE is NOT. I used to think that CORE tests are the minimal set of tests that are needed to evaluate the fitness for use of a record regardless of the use case (basically minimal set of tests that overlap any biodiversity use case), but I believe that it is not the case? (please correct me if I am wrong) For example, the newly added tests for pathway #277, #278

I don't know if these are helpful, please ignore if it doesn't. Thank you all SO MUCH for your hard work, I know time does not come cheap - I am so thankful to have the opportunity to work with you all!

ArthurChapman commented 6 months ago

Thanks @ymgan - great points and very helpful. One thing that springs to mind from your comments is that we can't document all use cases - if we followed the suggestions of @chicoreus, we would be making a random selection of a use case that certainly would not cover all cases. We currently have Examples that imply a use case, we cite where the test originated (ALA, VertNet, etc.)

@chicoreus - as said before TG3 was never meant to be comprehensive, but an exemplar or proof of concept. I attended all the early meetings of TG3 - in setting it up, and most of the meetings and discussions. It was a proof-of-concept and looking at how Use Cases could be developed and from that came the use of User Stories. Part way through, it was decided to link to the Framework and several were tested in conjunction with @allankv. TG3 was not comprehensive and was never intended to be comprehensive, and the majority of the TG2 tests were never covered by TG3. TG2 from the start was looking at a good set of tests, based on DwC and that would be "Fundamental tests of biodiversity data represented in Darwin Core terms that are widely applicable, informative, and straight forward to implement." We looked t what had been done by ALA, GBIF, iDigBio, CRIA, BISON, VertNet and others. There was never an idea of linking it directly to the Use Cases that came out of TG3. We had most of the TG2 tests prepared long before TG3 started to get any results.

For now we should just accept CORE as : "The set of mature tests that TG2 is putting forward as part of the standard. This is the bit meant by "Darwin Core terms that are widely applicable, informative, and straight forward to implement"

Perhaps, in the Document - we can have a section on adding future tests, that includes a workflow that includes documenting a Use Case, determining if it was "widely applicable, informative, and straight forward to implement" then follow the existing template, develop tests for implementation, then test implementation, etc.

Tasilee commented 6 months ago

Just back, briefly. I fully agree with @tucotuco and @ArthurChapman regards the circumscription of TG2 by TG3: Our tests are not bound by TG3 use cases. Our definition of core has been basically, as all have stated, "Tests that are widely applicable, informative, and straight forward to implement" with one exception: tests that we believe are 'aspirational' in encouraging a better best current practice (e.g., annotations).

I (strongly) believe that it is also informative that we define what is not CORE (out of scope of the standard) as it helps to clarify what is CORE, and document the environment to inform future uses. Thanks @ymgan for your comments. Our 'Supplementary', 'Immature/Incomplete' and 'Do not implement' are useful and are now adequately documented.

Like Arthur (as he well knows), I am also close to burnout on this work. We need to 'cut to the chase': Fill in gaps within the current CORE tests (e.g., test data - which I will do, and implementations) and get the standard document prepared.

ArthurChapman commented 5 months ago

Altered definitions of bdqtag: terms CORE, Supplementary, Immature/Incomplete, and DO NOT IMPLEMENT following recent discussions via email.

ArthurChapman commented 5 months ago

Added 5 new bdqffdq:UseCase terms for

bdq:Spatial-Temporal Patterns
bdq:Record-Management
bdq:Taxon-Management
bdq:Alien-Species
bdq:Biotic-Relationships

chicoreus commented 3 weeks ago

Most of the bdq:Response contexts aren't correct, here are a set of corrections to be applied:

namespace:Term	Context	Context Should Be
bdq:ASSUMEDDEFAULT	bdq:Response	bdqTestField:Term-Actions
bdq:CONVERTED	bdq:Response	bdqTestField:Term-Actions
bdq:ExpectedResponse	bdq:Response	bdq:Specification
bdq:FOUND	bdq:Response	bdqTestField:Term-Actions
bdq:OUTOFRANGE	bdq:Response	bdqTestField:Term-Actions
bdq:PRECISIONINSECONDS	bdq:Response	bdqTestField:Term-Actions
bdq:PREREQUISITESNOTMET	bdq:Response	bdqTestField:Term-Actions
bdq:PROPOSED	bdq:Response	bdqTestField:Term-Actions
bdq:Response	bdq:Response	bdq:Response
bdq:Response.comment	bdq:Response	bdq:Response
bdq:Response.qualifier	bdq:Response	bdq:Response
bdq:Response.result	bdq:Response	bdq:Response
bdq:Response.status	bdq:Response	bdq:Response
bdq:STANDARD	bdq:Response	bdqTestField:Term-Actions
bdq:STANDARDIZED	bdq:Response	bdqTestField:Term-Actions
bdq:TERRESTRIALMARINE	bdq:Response	bdqTestField:Term-Actions
bdq:TRANSPOSED	bdq:Response	bdqTestField:Term-Actions
bdq:CONSISTENT	bdq:Response.result	bdqTestField:Term-Actions

chicoreus commented 3 weeks ago

I've made an export of the vocabulary markdown table into https://github.com/tdwg/bdq/blob/master/tg2/vocabularies/combined_vocabulary.csv to give us something more easily sorted to look for inconsistencies and problems, and to start setting up to add vocabulary terms into various markdown documents.

As a demonstration of linking in vocabulary terms, I've added the definitions of the use cases to the index by use case section of: https://github.com/tdwg/bdq/blob/master/tg2/core/generation/docs/core_tests.md

For the time being, the markdown table in this issue remains the authoritative copy for editing, and we expect to overwrite the csv export.