ImperialCollegeLondon / virtual_ecosystem

This repository is the home for the codebase for the Virtual Ecosystem project.
https://virtual-ecosystem.readthedocs.io
BSD 3-Clause "New" or "Revised" License
8 stars 1 forks source link

Turning on single-threaded handling in dask #457

Closed davidorme closed 3 weeks ago

davidorme commented 3 weeks ago

Description

This PR is to fix segfaults in testing and docs building when xarray.open_mfdataset is used. This is due to underlying issues with the netCDF code not being thread safe. We previously turned on lock=False in that command, following advice here:

https://github.com/pydata/xarray/issues/3961

And that worked, but the issue has started occurring again. Following the advice in this issue thread:

https://github.com/pydata/xarray/issues/7079

this PR now explicitly sets the dask scheduler to use single-threading.

Fixes #354

Type of change

Key checklist

Further checks

codecov-commenter commented 3 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.75%. Comparing base (327ad85) to head (5ff2268).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #457 +/- ## ======================================== Coverage 94.75% 94.75% ======================================== Files 70 70 Lines 3791 3793 +2 ======================================== + Hits 3592 3594 +2 Misses 199 199 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.