Open taimoorsohail opened 3 months ago
cc'ing @dougiesquire here as well
so this is not an issue if you use conda envs 24.01 or earlier?
This is discussed on the Hive: https://forum.access-hive.org.au/t/xarray-warnings-while-loading-data-using-cosima-cookbook/2169/14
so this is not an issue if you use conda envs 24.01 or earlier?
Correct
@taimoorsohail does the threads_per_worker=1
kwarg in the dask Client solve this issue?
Hello,
I hit a Memory Error if I try to load a 3D variable (e.g., temperature) using cosima cookbook for analysis3-24.04 and greater.
I have isolated an example code that demonstrates the issue:
The error is:
MemoryError: Task 'getattr-d071745a-50f5-49b1-bda9-f8a50aeb807d' has 197.16 GiB worth of input dependencies, but worker tcp://127.0.0.1:37893 has memory_limit set to 35.88 GiB.
@angus-g??