search

ioos / glider-dac

The IOOS Glider DAC site/scripts/tools
http://gliders.ioos.us/providers/
7 stars 13 forks source link

address challenges encountered in implementing the QC test on the data arrays #318

Open leilabbb opened 3 months ago

leilabbb commented 3 months ago

This issue was created based on findings from working on #277

The QC process needs additional conditional statements to handle special cases of data arrays. The following have been already identified and are being tested:

Test 1: Check the time array for data start time inconsistent with the deployment start time, invalid timestamps, duplicate timestamps, and non-ascending timestamps Action 1: The QC tests are skipped and the Global attribute dac_qc_comment is annotated with test 1 results.

Test 2: Check the legacy variable for an empty array, an array with all fill values, or nans Action 2: The QC tests are skipped and the Global attribute dac_qc_comment is annotated with test 2 results.

Test 3: Check the legacy variables' standard names for non-CF compliant names Action 3: The QC tests are skipped and the Global attribute dac_qc_comment is annotated with test 3 results.

Test 4: Check the legacy variables for the missing attribute "ancillary_variables" Action 4: The QC tests are not skipped and the "ancillary_variables" is created

Test 5: Check variables sharing the same standard name as the legacy variables Action 5: The QC tests are skipped and the Global attribute dac_qc_comment is annotated with test 5 results.

leilabbb commented 3 months ago

Remark: Need a consistent way to parse the deployment start time to complete test 1. Parsing the deployment name using the specified date/time convention (YYYYmmddTHHMM) defined in the deployment registration guideline.