NCAS-CMS / cfdm

A Python reference implementation of the CF data model
http://ncas-cms.github.io/cfdm
MIT License
28 stars 11 forks source link

Allow access to netCDF-4 files in S3 object stores #285

Closed davidhassell closed 1 month ago

davidhassell commented 7 months ago

Currently netCDF files are only opened and read by the netCDF4 library. However, netCDF4 can't read files in S3 object stores.

h5netcdf, on the other hand can, access S3 netCDF-4 files (but not netCDF-3).

A new default behaviour for an arbitrary file should be to try netCDF4, and fall back on h5netcdf if that doesn't work. There should also be the option to cfdm.read to specifically choose one or other of the libraries.

Access to an S3 object store may require credentials, which will need to be passed in via a new cfdm.read argument.