I'm trying to understand and optimize the memory usage of Dask. For this simple example, I create an array that I would expect would occupy ~1GB of memory. When I realize the data with persist() this is what I find. But with compute() I see the amount…
I'm trying to understand and optimize the memory usage of Dask. For this simple example, I create an array that I would expect would occupy ~1GB of memory. When I realize the data with
persist()this is what I find. But withcompute()I see the amount…Would you like to know more?
Read the full article on the following website:
https://stackoverflow.com/questions/73770527/dask-compute-uses-twice-the-expected-memory