arviz-devs / arviz

Exploratory analysis of Bayesian models with Python
https://python.arviz.org
Apache License 2.0
1.56k stars 388 forks source link

h5py error when using `to_netcdf` #2296

Open jonsedar opened 7 months ago

jonsedar commented 7 months ago

Describe the bug

Really simple one I hope. When I try to write to_netcdf() I get

UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2, this may cause problems

This is using a conda-forge based install

Crosspost from https://github.com/conda-forge/h5py-feedstock/issues/122

To Reproduce

Really simple

idata.to_netcdf('filename.netcdf')

Additional context

Python implementation: CPython
Python version       : 3.10.13
IPython version      : 8.18.1

pymc    : 5.9.2
pytensor: 2.17.3

Compiler    : Clang 16.0.6 
OS          : Darwin
Release     : 23.1.0
Machine     : arm64
Processor   : arm
CPU cores   : 8
Architecture: 64bit

seaborn   : 0.12.2
pandas    : 2.1.4
arviz     : 0.16.1
numpy     : 1.25.2
oreum_core: 0.7.0
sys       : 3.10.13 | packaged by conda-forge | (main, Oct 26 2023, 18:09:17) [Clang 16.0.6 ]
OriolAbril commented 6 months ago

Is this on a fresh env?

I don't really see what can be done on ArviZ side other than waiting for the linked issue to be solved. For older arviz versions (like the one in the original issue) the situation is a bit trickier if you want to use h5py because both h5py and netcdf4 depend directly on hdf5 and so even using conda, they might be "fighting" each other and trying to use different hdf5 versions.

However, we recently switched from netcdf4 to h5netcdf (which depends on h5py and through that on hdf5) so unless you manually install hdf5 or netcdf4 there should never be multiple hdf5 versions around.

Note: h5py has roughly 10 times more downloads on pypi, and there are many libraries that depend on it, so having h5py+older-arviz installed in an env should be much more common than having newer-arviz+netcdf4