Remove recipe filler utility

[schlunma]

Description

With the introduction of the wildcard syntax for recipes in ESMValGroup/ESMValCore#1609 the recipe filler utility has become obsolete. After parsing a recipe, ESMValTool will automatically create a "filled" version of the recipe (available in the run output directory) with all wildcards expanded, which can then be modified by the user and run again.

Details on how to use the new wildcard syntax: https://docs.esmvaltool.org/projects/ESMValCore/en/latest/recipe/overview.html#automatically-populating-a-recipe-with-all-available-datasets

Example notebook: https://github.com/ESMValGroup/ESMValCore/blob/main/notebooks/discovering-data.ipynb

---

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• [x] 🛠 This pull request has a descriptive title
• [x] 🛠 Code is written according to the code quality guidelines
• [x] 🛠 Documentation is available
• [x] 🛠 Tests run successfully
• [x] 🛠 The list of authors is up to date
• [x] 🛠 Any changed dependencies have been added or removed correctly
• [x] 🛠 All checks below this pull request were successful

---

To help with the number of pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[schlunma]

Don't worry, I would never merge this without proper discussion! Sorry, opening this PR takes as much time as opening an issue, so I thought I might as well do that!

Regarding the features:

note that the wildcard approach is not exactly equivalent with the recipe filler: that will find and run everything that is available, whereas the recipe filler will tell the user what is available before the actual run, then the user will decide what data to run with

This can also be achieved with the wildcard syntax. For every ESMValTool run, a filled recipe is written which, in case wildcards are used, contains all expanded datasets. User can modify this as they wish to achieve the same functionality as the recipe filler.

Also, the wildcard syntax fixes some of the limitations of the recipe filler:

the tool doesn’t yet work with derived variables; it will not return any available datasets; operation restricted to CMIP data only, OBS lookup is not available yet.

[valeriupredoi]

Oh that's very nice about the filled recipe! Didn't know about that feature - that's that sold to me, in this case, but let's still ask some of the users, not being a user per se myself, am not aware of potential used cases users may be employing the recipe filler for. Thanks, Manu 🍺

[egalytska]

That all sounds good to me. However, there is one minor drawback in my case. In the paper I wrote that I used recipe_filler to get the info regarding available variables from the models. If someone by any miracle decides to search for this recipe filler, they obviously won't find it. This can be quite annoying. As a workaround, would it be possible to mention that recipe_filler is deprecated and wildcard syntax for recipes has the same (or improved) functionality?
Best regards to everyone :sparkling_heart:

[valeriupredoi]

@egalytska indeed we'll mention that as you say, in the 2.12 release notes, and the recipe filler still lives in history ie in 2.11 and before 🍺

[schlunma]

@ESMValGroup/esmvaltool-developmentteam In this pull request we plan to remove the recipe filler utility because it has become obsolete (see details on the pull request description above). If you are a regular user of this tool and/or think we should not remove it, please let us know as soon as possible by commenting here. Thanks!!

[valeriupredoi]

@esmvalbot please run examples/recipe_python.yml

[esmvalbot[bot]]

Since @valeriupredoi asked, ESMValBot will run recipe examples/recipe_python.yml as soon as possible, output will be generated here

[esmvalbot[bot]]

ESMValBot is sorry to report it failed to run recipe examples/recipe_python.yml: exit is 1, output has been generated here

[schlunma]

It looks like no one disagrees with this, so I guess we can safely go ahead with this PR! @valeriupredoi if you agree, would you please merge this? Thanks! 🍻

[valeriupredoi]

devel tests are with iris=3.11 and those fails are fixed by https://github.com/ESMValGroup/ESMValTool/pull/3810