cedadev / checksit

File-checking made simple
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Added datasets to the configuration #4

Open agstephens opened 2 years ago

agstephens commented 2 years ago

@spepler: I am thinking about a simple initial implementation on a per-dataset basis:

$ cat etc/checker.ini

[dataset:ukmo-assim]
regex_file = ukmo-nwp-strat_gbl-std_\d{10}_u-v-gph-t-w\.pp
template = /badc/ukmo-assim/data/standard/2022/ukmo-nwp-strat_gbl-std_2022010112_u-v-gph-t-w.pp

[dataset:ukcp18]
regex_path = ukcp18
template_cache = template-cache

[dataset:ukcp09]
regex_path = ukcp09
template = /badc/ukcp09/data/gridded-land-obs/gridded-land-obs-monthly/grid/ascii/rainfall/2016/ukcp09_gridded-land-obs-monthly_5km_rainfall_201601.txt

When you throw a new file at the checker, it goes through the dataset configurations, testing the regex on the path OR file. When it matches, it then does one of:

If no datasets are found then it will: