search

ncasuk / amf-check-writer

Library to write AMF compliance checks
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

Ensure that the downloader and parser report any errors clearly at the end of the process #2

Closed joesingo closed 3 years ago

joesingo commented 6 years ago

Old

agstephens commented 3 years ago

NOTE: the create yaml and CSV file scripts need to be made properly aware of dataset versions.

This should be more explicit so you don't have to set the output paths.

Everything should be written and read to/from a standard location, such as

$D/<version>/product-definitions/spreadsheet/ # For: ncasuk/AMF_CVs/product-definitions/spreadsheet/ 
$D/<version>/product-definitions/tsv/ # For: ncasuk/AMF_CVs/product-definitions/tsv/ 
$D/<version>/AMF_CVs/  # For: ncasuk/AMF_CVs/AMF_CVs
$D/<version>/checks/  # For: ncasuk/amf-compliance-checks/
$D/<version>/pyessv-vocabs/ # For: ncasuk/AMF_CVs/pyessv-vocabs

And maybe we only provide the output base directory to each script, and a separate version parameter, that would dictate <version> and put everything in the right place.

@gapintheclouds: does this make sense to you?

agstephens commented 3 years ago

NOTE: we are now starting to put the information about the process here:

https://github.com/ncasuk/amf-check-writer/blob/master/workflow_data.yml

This means we can validate the content found on the google-drive (and when processing) against this content.

agstephens commented 3 years ago

Duplicates other info.