Change default `xarray_open_kwargs` to use `chunks="auto"`

dougiesquire commented 8 months ago

Is your feature request related to a problem? Please describe. Currently, by default, Intake-ESM opens files using chunks={}. For netCDF files:

with xarray < v2023.09.0, this gives one chunk per file, which is sometimes a bad choice;
with xarray >= v2023.09.0, this gives the file chunking, which is often a bad choice.

Describe the solution you'd like Default to chunks="auto" to use dask auto chunking. This should give better chunking in most cases.

Happy to submit a PR.

Describe alternatives you've considered Passing the kwarg every time I use Intake-ESM

dcherian commented 8 months ago

gives the file chunking

Isn't this a multiple of the file chunking?

dougiesquire commented 8 months ago

I think chunks="auto" gives multiples of the file chunks, but chunks={} gives the file chunks.

mgrover1 commented 8 months ago

This sounds reasonable to me... feel free to go ahead and open a PR!

dougiesquire commented 8 months ago

It was decided to leave the default as chunks={} since chunks="auto" does not work with object dtype (which is common).

intake / intake-esm