search

ioos / ioos_qc

:ballot_box_with_check: :ocean: IOOS QARTOD and other Quality Control tests implemented in Python
https://ioos.github.io/ioos_qc/
Apache License 2.0
46 stars 27 forks source link

use h5netcdf instead of netCDF4 #71

Closed ocefpaf closed 1 year ago

ocefpaf commented 2 years ago

This is the first step to reduce the dependency on complex binaries. The h5netcdf library uses h5py to mimic netCDF4's API and, b/c wheels for it are more readily available we can increase the options where we can install ioos_qc. See https://github.com/h5netcdf/h5netcdf for more information on the advantages not depending on netcdf-c.

PS: h5netcdf is used by many Earth Sciences communities, including xarray, so it will be properly maintained and the switch should not bring any headaches.

ocefpaf commented 2 years ago

The failures here seems to be unrelated to this PR and hopefully will be fixed in #70

kwilcox commented 2 years ago

I'm on board but let's hold this until the other PRs are merged, I will add some corner case tests and try to break things.

kmuehlbauer commented 2 years ago

PS: h5netcdf is used by many Earth Sciences communities, including xarray, so it will be properly maintained and the switch should not bring any headaches.

h5netcdf maintainer here. Please report any issues you might discover on the way. Trying to keep any headaches as short as possible :wink:

ocefpaf commented 2 years ago

@kmuehlbauer I've postponed a lot the move to h5netcdf, mostly b/c I was lazy. Lately I'm using it pretty much everywhere without any issues. The main reason that forced me to move is the ability to use it with pyscript, hdf5 and h5py have larger communities than libnetcdf and netcdf4-python, so they are readily available everywhere., That makes h5netcdf the easiest way to read/write netcdf files with pyscript.

kwilcox commented 1 year ago

Merged manually after a rebase, this had some leftover changes from the xarray change branch: https://github.com/ioos/ioos_qc/commit/06f401d3cd44886fd639bf6f1bf3bea623c43a00