Open bouweandela opened 5 years ago
Other error messages and checks should be also be improved, e.g. when you provide a caption that is a list instead of a string, you will get this rather cryptic error message:
"""
Traceback (most recent call last):
File "/mnt/lustre01/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 671, in _run_task
output_files = task.run()
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 227, in run
self.output_files = self._run(input_files)
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 496, in _run
self._collect_provenance()
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 559, in _collect_provenance
product.save_provenance()
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_provenance.py", line 252, in save_provenance
self._include_provenance()
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_provenance.py", line 248, in _include_provenance
write(filename, attributes)
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_provenance.py", line 219, in _include_provenance_nc
setattr(dataset, key, value)
File "netCDF4/_netCDF4.pyx", line 2882, in netCDF4._netCDF4.Dataset.__setattr__
File "netCDF4/_netCDF4.pyx", line 2812, in netCDF4._netCDF4.Dataset.setncattr
File "netCDF4/_netCDF4.pyx", line 1568, in netCDF4._netCDF4._set_att
OSError: array string attributes can only be written with NETCDF4
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_main.py", line 228, in run
conf = main(args)
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_main.py", line 156, in main
process_recipe(recipe_file=recipe, config_user=cfg)
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_main.py", line 206, in process_recipe
recipe.run()
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_recipe.py", line 1190, in run
self.tasks, max_parallel_tasks=self._cfg['max_parallel_tasks'])
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 596, in run_tasks
_run_tasks_parallel(tasks, max_parallel_tasks)
File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 641, in _run_tasks_parallel
task.output_files, updated_products = result.get()
File "/mnt/lustre01/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
OSError: array string attributes can only be written with NETCDF4
2019-09-05 14:34:23,779 UTC [23512] INFO If you suspect this is a bug or need help, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues and attach the run/recipe_*.yml and run/main_log_debug.txt files from the output directory.
The plan for the next release will be to implement the schema and only allow a single correct type for each entry.
Note that currently there is a warning if no ancestor files are provided, however, this is probably a valid use case if the file is generated from code alone: https://github.com/ESMValGroup/ESMValTool/pull/1722#issuecomment-660959216, so this should be supported.
It would also be nice to add provenance for ancestor files from auxiliary_data_dir
(and drop the warnings if these are used).
At the moment diagnostic developers either get no error message at all, or various obscure error messages if they make a mistake in their diagnostic_provenance.yml file. It would be nicer if we could get a more clear error message by using a Yamale schema to check it.