Set resources for RTW recipes individually

alistairsellar commented 6 months ago

Description

List all recipes under the same cylc/jinja variable, and set resources individually, with actual example usage captured in comments in the site/*.cylc files. Cheap recipes can use the default resources, with everything else listed explicitly.

Closes #3561
Link to documentation: https://esmvaltool--3562.org.readthedocs.build/en/3562/
- *

Before you get started

[x] ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[x] 🛠 Code is written according to the code quality guidelines
[x] 🛠 Documentation is available
[x] 🛠 Tests run successfully
[x] 🛠 The list of authors is up to date
[x] 🛠 Any changed dependencies have been added or removed correctly
[x] 🛠 All checks below this pull request were successful

alistairsellar commented 6 months ago

Hi @ehogan, I implemented step 1 of what we discussed last week - how the processing resources are specified. Please could you review?

alistairsellar commented 6 months ago

Now done and tested on Met Office and Jasmin.

ehogan commented 5 months ago

Ah, just noticed the error message slurmstepd: error: Exceeded step memory limit at some point. for process_albedolandcover, so the memory for this task may need increasing slightly 👍

alistairsellar commented 5 months ago

Ah, just noticed the error message slurmstepd: error: Exceeded step memory limit at some point. for process_albedolandcover, so the memory for this task may need increasing slightly 👍

Good spot. Added in dc377b9

alistairsellar commented 5 months ago

Looks great, thanks @alistairsellar! 🎉

One super picky request: would it be possible to add some periods to the end of the comments you added, please? 😊

Done in 2cb2e54

alistairsellar commented 5 months ago

Thanks for the review @ehogan. I think I've addressed all your comments. Please can you take another look?

alistairsellar commented 5 months ago

Great, thanks @ehogan!

ESMValGroup / ESMValTool