Closed SerRichard closed 2 years ago
This boolean needs to be switched, I only just noticed but having it set to true seems to pull the entire dataset into client memory, we really want to avoid that, setting it false seems to keep it dask side. Was not obvious this was happening!
Ah, interesting - lgtm!
This boolean needs to be switched, I only just noticed but having it set to true seems to pull the entire dataset into client memory, we really want to avoid that, setting it false seems to keep it dask side. Was not obvious this was happening!