Allow usage of relative paths for diagnostic scripts

[rbeucher]

Hi everyone,

Currently, in the diagnostic section of a recipe, the script must be either:

• A valid path to an existing diagnostic shipped with ESMValtool (diagnostic root)
• An absolute path to a diagnostic (not portable)

Looking at the code here, there is a third option (not sure that one is very useful though):

https://github.com/ESMValGroup/ESMValCore/blob/6f879cfbff20568ca6b9e2ea4667793a44bef2e5/esmvalcore/_task.py#L350C1-L358C70

It appears that things like ~/my_esmvaltool_diagnostics/my_diagnostic.py will expand to a valid location.

Absolute paths are not portable... We are currently developing some material to teach people how to develop their diagnostics. We would like to be able to ship a recipe.yml and a diagnostic.py together, with the recipe referencing the diagnostic using a relative path.

I propose to check for a local diagnostic script before appending the esmvaltool diagnostic root to the path. PR coming.

[bouweandela]

Note that there are actually several paths needed for a successful run: 1) location of esmvaltool/diag_scripts 1) location of esmvaltool/config-references.yml 1) location of esmvaltool/references 1) location of esmvaltool/recipes (if a relative path to a recipe is provided) 1) location of esmvaltool/interface_scripts (only for NCL diagnostics)

Maybe the best solution would be to add a new configuration option diagnostic_package_dir or something similar to the configuration file/command line and make that point to the installed esmvaltool package by default with a fallback to the current working directory.

See #826 for some previous, slightly outdated, discussion.