ESMValGroup / ESMValCore

ESMValCore: A community tool for pre-processing data from Earth system models in CMIP and running analysis scripts.
https://www.esmvaltool.org
Apache License 2.0
42 stars 38 forks source link

Avoid a crash when there is a timeout when shutting down the Dask cluster #2580

Open bouweandela opened 1 week ago

bouweandela commented 1 week ago

Description

Avoid a crash when there is a timeout when shutting down the Dask cluster. I have seen this happen on Levante at DKRZ recently. It does not appear to be a real problem as the cluster is shut down after the tool completes running.


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the ๐Ÿ›  Technical or ๐Ÿงช Scientific review.


To help with the number pull requests:

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.66%. Comparing base (a328578) to head (be726ed).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2580 +/- ## ======================================= Coverage 94.66% 94.66% ======================================= Files 251 251 Lines 14287 14290 +3 ======================================= + Hits 13525 13528 +3 Misses 762 762 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

bouweandela commented 4 days ago

How about "Timeout while trying to shut down the cluster at %s, you may want to check it was stopped."?

valeriupredoi commented 1 day ago

How about "Timeout while trying to shut down the cluster at %s, you may want to check it was stopped."?

sounds soothing enough :grin: