ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
https://www.esmvaltool.org
Apache License 2.0
227 stars 128 forks source link

Add an option to disable using the distributed scheduler from the diagnostic script #3787

Open bouweandela opened 1 month ago

bouweandela commented 1 month ago

Description

Some diagnostic scripts do not work with the distributed scheduler. Typically this happens because they insert large numpy arrays into the task graph, which subsequently leads to the scheduler killing the worker because they exceed their maximum allowed amount of memory. This adds an option to use the default scheduler for some diagnostic scripts by specifying no_distributed: true in the diagnostic script settings.

The diagnostic script in recipe_eady_growth_rate.yml does not work with the distributed scheduler. This pull request disables using the distributed scheduler for that recipe.

Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the πŸ›  Technical or πŸ§ͺ Scientific review.


To help with the number of pull requests: