TG2-TO DO List for standard submission

A list of things for @ArthurChapman, @Tasilee, @tucotuco, @chicoreus and @ymgan to complete the BDQ Core standard submission

Introduction Document and Landing Pages

(Done) Replicate target directory structure for the standard within _review @tucotuco
(Done) Files under _review directory that require some manual content @chicoreus (content parsed out of supplemental into users guides and other documents).
(Done) Diagrams showing a generic approach to using Validations, Measures, Amendments, Validations, Measures @ymgan
- (Done) Place diagram svg files within _review directory. in _review/docs/intro @chicoreus
(Done) Modfications to target directory structure to include derivative files for review @chicoreus See: _review/README for current state of files and directories
(Done) Identify files that are currently maintained outside of _review and copied into it. @chicoreus see: https://github.com/tdwg/bdq/blob/master/tg2/_make_review/copy_files.sh
Pare down Introduction document to minimum normative and non-normative needed @tucotuco (done through section 1)
(Done) Add landing pad pages for bdqffdq (required), and bdqcore (desirable). @chicoreus
- (Done) Create bdqcore landing page. @chicoreus
- (Done) Refine which text goes on bdqcore landing page and which on bdqcore term-list @Tasilee @chicoreus
- (Done) Create bdqffdq landing page. @chicoreus
- (Done) Add normative guidance about use of bdqffdq terms to bdqffdq landing page. @chicoreus
(Done) Add list of documents with target audience for each to end of introduction @chicoreus

Supplementary Document

(Done) Edit Supplementary Document @Tasilee
(Done) Put supplementary information into supplementary document @Tasilee
(Done) Move normative information from supplementary document to appropriate other documents, limit content of supplementary to just supplementary. @chicoreus
(Done) Format supplementary document @ArthurChapman
(Done) Copy the bdqTag vocabulary into a markdown table in the supplementary document and remove from vocabulary/glossary_terms.csv @chicoreus
(Done) Copy the bdqTestField vocabulary into a markdown table in the supplementary document and remove from vocabulary/glossary_terms.csv. @chicoreus
(Done) Add history of the development of the tests to Section 3.
(Done) Merge the three duplicative sets of Test Field descriptions in the supplementary documentation in to one.
(Done) Add Competency Questions section for bdqffdq and bdqcore to the supplementary documentation. @chicoreus
Write additional sparql queries on bdqcore RDF and use these as competency questions in supplement @chicoreus
Done) Review Supplement Document @ArthurChapman
(Done) Review document @Tasilee
Add discussion of bdq_issue_to_csv, bdqtestrunner, and kurator-ffdq to supplement. @chicoreus

Vocabularies and termlist documents

(Done) Flesh out Vocabulary landing page @ArthurChapman, @tucotuco
(Done) Review which terms go where @ArthurChapman @chicoreus
(Done) Review bdq and bdqdim term definitions @ArthurChapman
(Done) Update bdqffdq: Ontology @chicoreus @ArthurChapman
- (Done) Review bdq/bdqffdq vocabulary definitions @ArthurChapman @chicoreus
- (Done) Move ontology into tdw/bdq respository @chicoreus
- (Done) Update bdqffdq ontology skos:definitions and skos:comments @chicoreus
- (Done) Give each bdqffdq term a skos:prefLabel, skos:definition, rdfs:label, and rdfs:comment @chicoreus
- (Done) Create csv export of ontology @chicoreus (note, needs manual edit with each export from owl).
- (Done) Integrate First draft Python code to produce human readable display of ontology into _review/build/ , done as draft_build_bdqffdq.py @chicoreus
- (Done) Add Argument and related terms for Parameters and Arguments on Specifications to bdqffdq @chicoreus
Provide rdf distributions of Vocabulary Term List documents following SDS section 2.2.4. (owl document for bdqffdq)
- (Done) rdf artifacts for Vocabulary Terms lists of bdq, bdqdim, bdqenh, bdqcrit @chicoreus
- Update rdf artifact for bdqcore generated with kurator-ffdq @chicoreus
  - (Done) Multiple tests in each Policy @chicoreus,
  - (Done) Class names matched to bdqffdq. @chicoreus
  - (Done) Add Criterion, Enhancement, DataQualityDimension instances from bdqcrit, bdqenh, bdqdim vocabularies @chicoreus
  - (Done) Stable identifiers for methods, specifications, information elements in RDF generated from CSV @chicoreus (tg2/core/information_element_guids.csv and tg2/core/TG2_tests_additional_guids.csv used by kurator-ffdq with -guidFile and -ieGuidFile options)
  - (Done) Provide for reuse of uuids between builds of RDF artifact from configuration files @chicoreus
  - (Done) stable identifier file for core test methods/specifications/profiles @chicoreus
  - (Done) stable identifier file for information elements @chicoreus
  - (Done) add stable identifiers for methods/specifications/profiles for multirecord measures @chicoreus
  - Add Arguments and Parameters to bdqcore RDF serialization @chicoreus
  - Add and use stable identifier file for arguments @chicoreus
(Done) Add a vocabulary extension file for bdqffdq @chicoreus
Build Vocabulary term list files in _review/docs/list/{term}/index.md files @chicoreus
- (Done) Build script for term list files modified to run with local configuration files for draft standard @chicoreus
- (Done) Configure and run for bdq, bdqdim, bdqenh, bdqcrit vocabularies. @chicoreus
- Update Vocabulary Landing page
- links to point to new location for final term list documents rather than termlist header documents, when the former are ready @tucotuco
(Done) Review human readable term-list files in
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdq/index.md @ArthurChapman
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqdim/index.md @ArthurChapman
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqcrit/index.md @ArthurChapman
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqenh/index.md @ArthurChapman
- (Done) Checked for consistency of headings between all four documents. Header files updated @ArthurChapman @chicoreus
(Done) Document and close Issue #152 as this vocabulary has been split into separate files and added to _review @ArthurChapman
Add remaining glossary items from #152 that aren't in vocabularies into glossary section of introduction @chicoreus
(Done) Add vocabularies and pages for terms to apply to Criterion and Enancement fundamental concepts @chicoreus
(Done) Hold discussion with Cecilie Svenningsen (GBIF) on Vocabularies @ArthurChapman
(Done) Provide GBIF with list of future Vocabularies needed for Data Quality @ArthurChapman
(Done) Review bdq term list header document @Tasilee**
How will Section 1.5 of the 'BDQ Core Tests and Assertions List of Terms' document be handled? We either link (probably) to the terms used in the Tests listed below, or pull in a term table from relevant vocabs. @chicoreus, @ArthurChapman, @Tasilee, @tucotuco

Quick Reference Guide

(Done) Create Preface for the Quick Reference Guide @Tasilee
(Done) Decide what descriptors that will be in the Quick Reference Guide @Tasilee
(Done) Build bdqcore Quick Reference Guide @Tasilee
(Done) Create draft QRG Footer (Citation based on Darwin Core QRG) @Tasilee
(Done) Python script to produce [quick reference guide] @chicoreus (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/terms/bdqcore/index.md) for combined singlerecord and multirecord bdqcore tests @chicoreus .
(Done) Incorporate this script with header/footer assembly @tucotuco @chicoreus

Users Guide

(Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
Review content to confirm no missing assumptions in understanding and clarity of text for consumers.
(Done)Generate TOC (sections for populating) @chicoreus
allocate sections to team members (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/users/index.md) @chicoreus
(Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus
(Done) Review/edit document @Tasilee

Implementation Guide

(Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
(Done) Generate TOC (sections for populating) @chicoreus
allocate sections to team members (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/implementers/index.md)
(Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus
(Done) Review document @Tasilee
Add note on stub code generation by kurator-ffdq @chicoreus

bdqffdq Guide

(Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
(Done) Create visualizations to explain ontology, add to bdqffdq guide page. @chicoreus (files are in _review/build/template/guide/bdqffdq/)
Edit the above ontology visualizations @ArthurChapman (in progress)
(Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus

Standards Maintenance Document

(Done) Stub for Standard Maintenance document
(Done) Add a section on Test Data @Tasilee
Add a section on supporting code (bdq_issue_to_csv, bdqtestrunner, kurator-ffdq) @chicoreus

Tests

(Done) Review/edit (English) Description in the light of current Expected Responses (CORE) @ArthurChapman
(Done) Add skos:prefLabel for multi-record measures in bdq_issue_to_csv @chicoreus
Update and add links to source code for implementations in test descriptors for filtered push libraries @chicoreus
Freeze editing of markdown table representations of tests (decision by all)
- Final export from markdown tables with bdq_issue_to_csv @chicoreus
- Then: Update rdf artifact for bdqcore generated with kurator-ffdq @chicoreus
- (Done) Replace all instances of EMPTY and NOT EMPTY, etc. in the test specifications with bdq:Empty @Tasilee @ArthurChapman
(Done) Build script that copies latest validation data files (test data and non-printing characters) into _review directory. @chicoreus See _make_review/copy_files.sh
(Done) Add implementation for #287 run against validation data. @chicoreus
(Done) Check generated Examples in Test Data match those in the Tests and align accordingly @Tasilee
(Done) Check and align inconsistencies in phrasing in the Specification of "in bdq:sourceAuthority" vs "in the bdq:sourceAuthority" @Tasilee
(Done) Check consistency of use of bdq:sourceAuthority and ISO references in all tests and align @Tasilee
(Done) Complete review of FAIL cases in validation data, update test specifications, validation data, or implementation as needed (also under Test Data below) @chicoreus @Tasilee @ArthurChapman Filteredpush implementations are passing all validation rows as of v93

References

(Done) Prepare intro for References @Tasilee
- (Done) Format RFC 2119 reference and include in reference list @Tasilee
(Done) @Tasilee to send @ArthurChapman a list of References that need correction
(Done) Correct References in the Test Issues @ArthurChapman, @Tasilee
(Done) Prepare a combined list of References @Tasilee
(Done) Update with additional new references @Tasilee
New releases of related code, update references with links to zenodo, make sure each piece of software has text in the appropriate document with a citation. @chicoreus

Authors and Contributors

(Done) Prepare file on Authors and Contributions @chicoreus, @Tasilee, @ArthurChapman
(Done) Add code library functions to insert authors/contributors sections in consistent form into templates @chicoreus .
(Done) Chase up missing authors and ORCID_IDs @ArthurChapman @chicoreus
Decide on breakdown of Authors, Contributors and Acknowledgements @tucotuco, @ArthurChapman, @Tasilee, @chicoreus
Add configuration file for acknowledgements? Seems a good idea that works with authors configuration file. In progress @chicoreus @ArthurChapman @Tasilee

Test Data

(Done) Add new column for 'Date Last Updated' and populate with current date @Tasilee
(Done) Rationalize (align for consistency) of Response.comments and align with Test Examples (e.g., tense, phrasing) @Tasilee
(Done) Complete review of FAIL cases in validation data, update test specifications, validation data, or implementation as needed @chicoreus @Tasilee @ArthurChapman
- (Done) Resolve 3 apparent duplicate rows @Tasilee (in v91, fixed in v93) Note: All filtered push qc implementations passing all validation rows except the 3 that appear as duplicates, and the annotation test for which there is no implementation.

Consistency Checks

(Done) Align (2) full and (2) summary sections about Test Types across 4 documents @Tasilee
(Done) Review and make consistent descriptive text about test types on bdqcore landing page, bdqcore term-list page, supplement, users guide, implementers guide. @Tasilee @chicoreus
Refine descriptive text about test types to match needs of local context and add normative statements where appropriate to bdqcore landing page, bdqcore term-list page, users guide, and implementers guide. @Tasilee @chicoreus
Consistent Headings
- Purpose and Audience statements for each document (@chicoreus added), needs refinement.
- Consistent parts and ordering for section 1 of documents, following expectations of SDS (first pass by @chicoreus done).

General

(Done) Identify potential Review Managers
Identify potential Maintenance Managers
(Done) Prepare and send off Report to Executive (All)
(Done) Make verbal report and summary to TDWG Business Meeting at TDWG2024 @ArthurChapman
(Done) Prepare diagrams for each of the test types (NAME, SPACE, TIME, OTHER) showing linkages between the tests @ArthurChapman
(Done) Templates and build scripts for all /docs/ pages except landing page for standard, maintinance document, and references page. @chicoreus
produce RDF triples from the diagrams using showing linkages between the tests using improvedBy, targetedIssue, targetedMeasure, targetedValidation terms (In progress) @ArthurChapman ??
Final (editorial) review of all documents :) @tucotuco

tdwg / bdq

TG2-TO DO List for standard submission #300

Introduction Document and Landing Pages

Supplementary Document

Vocabularies and termlist documents

Quick Reference Guide

Users Guide

Implementation Guide

bdqffdq Guide

Standards Maintenance Document

Tests

References

Authors and Contributors

Test Data

Consistency Checks

General