NCAS-CMS / cf-python

A CF-compliant Earth Science data analysis library
http://ncas-cms.github.io/cf-python
MIT License
120 stars 19 forks source link

Allow access to netCDF-4 files in S3 object stores #712

Open davidhassell opened 7 months ago

davidhassell commented 7 months ago

Currently netCDF files are only opened and read by the netCDF4 library. However, netCDF4 can't read files in S3 object stores.

h5netcdf, on the other hand can, access S3 netCDF-4 files (but not netCDF-3).

A new default behaviour for an arbitrary file should be to try netCDF4, and fall back on h5netcdf if that doesn't work. There should also be the option to cf.read to specifically choose one or other of the libraries.

Access to an S3 object store may require credentials, which will need to be passed in via a new cf.read argument.