Here are some improvements and explanation on Dask.
I also fixed some code that wouldn't work near the end (either too heavy on memory, either wrong variable used).
I'd like also to discuss some points:
Warning on spilling, I think this is too complex to introduce yet.
dask.optimize(): should we really introduce this? This makes a different in number of tasks displayed, but upon execution, Dask is clever enough to not execute the useless tasks, I tested with and without optimization on graph, and the result is almost the same. I'm under the impression that Dask does optimize things anyway upon calling compute.
Here are some improvements and explanation on Dask.
I also fixed some code that wouldn't work near the end (either too heavy on memory, either wrong variable used).
I'd like also to discuss some points:
Fixes #50.