/home/eczech/miniconda3/envs/sgkit-dev/lib/python3.8/site-packages/xarray/core/indexing.py:1361: PerformanceWarning:
Slicing is producing a large chunk. To accept the large
chunk and silence this warning, set the option
>>> with dask.config.set(**{'array.slicing.split_large_chunks': False}):
... array[indexer]
To avoid creating the large chunks, set the option
>>> with dask.config.set(**{'array.slicing.split_large_chunks': True}):
... array[indexer]
return self.array[key]
We should figure out how this is possible when the functions applied to a dataset do nothing other than filter within chunks. Presumably this means the chunks should only shrink unlike what is suggested in the warning.
I haven't been able to reproduce this on simulated data yet.
I see this warning when running the function mentioned in https://github.com/pystatgen/sgkit/issues/299 on 1KG data:
We should figure out how this is possible when the functions applied to a dataset do nothing other than filter within chunks. Presumably this means the chunks should only shrink unlike what is suggested in the warning.
I haven't been able to reproduce this on simulated data yet.