This PR adds a vars argument to generate_batch_files.py and run_generate_batch_files.py. There is a corresponding PR for the var_choicebranch in the prefect repo.
The generate_batch_files.py script now iterates thru a list of variables while searching for *.nc files in the CMIP6 directory. This process is done before the read_grids() function and therefore filters out unwanted variables before touching the files.
TO TEST:
Run the regridding Prefect flow as usual, using the var_choice branch in the prefect repo and specifying the var_choice branch of this repo in the Prefect run. Using the new {vars} parameter input, try:
a short list of valid variables (e.g., "pr ta tas")
one of the pre-defined lists (e.g., "land")
In both tests, run the flow up until the batch files have been generated. At that point you can cancel the flow and check out a random subset of the batch files to make sure you don't see any other variables in the batches. Be sure to delete batch files between tests!
try a bad list (e.g., "foo bar") and make sure the flow fails
review the pre-defined lists in the luts.py file of the prefect repo var_choice branch. Does this look right? Is anything missing or misplaced?
This PR adds a
varsargument togenerate_batch_files.pyandrun_generate_batch_files.py. There is a corresponding PR for thevar_choicebranch in the prefect repo.The
generate_batch_files.pyscript now iterates thru a list of variables while searching for *.nc files in the CMIP6 directory. This process is done before theread_grids()function and therefore filters out unwanted variables before touching the files.TO TEST:
Run the regridding Prefect flow as usual, using the
var_choicebranch in the prefect repo and specifying thevar_choicebranch of this repo in the Prefect run. Using the new{vars}parameter input, try:In both tests, run the flow up until the batch files have been generated. At that point you can cancel the flow and check out a random subset of the batch files to make sure you don't see any other variables in the batches. Be sure to delete batch files between tests!
luts.pyfile of the prefect repovar_choicebranch. Does this look right? Is anything missing or misplaced?