search

ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
https://www.esmvaltool.org
Apache License 2.0
227 stars 128 forks source link

recipe_autoassess_stratosphere produces very large output: add option to remove it when finished #2376

Open bouweandela opened 3 years ago

bouweandela commented 3 years ago

The output of most recipes is fairly small (in the order of megabytes), but recipe_autoassess_stratosphere.yml produces an output of 14GB. This is very large, is this needed? In particular the files

334M    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/OBS/ERA-Interim_cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/MPI-ESM-MR/stratosphere/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/MPI-ESM-MR/stratosphere_supermeans/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/MPI-ESM-LR/stratosphere/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/MPI-ESM-LR/stratosphere_supermeans/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/inmcm4/stratosphere/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/inmcm4/stratosphere_supermeans/cubeList.nc
334M    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/ERA-Interim/stratosphere/cubeList.nc
334M    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_1/ERA-Interim/stratosphere_supermeans/cubeList.nc
334M    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/OBS/ERA-Interim_cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/MPI-ESM-LR/stratosphere/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/MPI-ESM-LR/stratosphere_supermeans/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/inmcm4/stratosphere/cubeList.nc
1.2G    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/inmcm4/stratosphere_supermeans/cubeList.nc
334M    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/ERA-Interim/stratosphere/cubeList.nc
334M    recipe_autoassess_stratosphere_20211023_205958/work/aa_strato/autoassess_strato_test_2/ERA-Interim/stratosphere_supermeans/cubeList.nc

are very large. Is this a result or are this just some intermediate files?

valeriupredoi commented 3 years ago

Cheers @bouweandela - these are intermediate files that can be deleted after the run has finished successfully, I think there is an option to do that, lemme look it up

valeriupredoi commented 3 years ago

nope no such option, needs implementation - hence changing issue's title :+1: