Bug report: cyclic dependencies in h5 python bindings

pavel-po commented 9 months ago

Description

With a high version of python3, I get error like this one using h5:

  File "/mnt/home/ppokhilko/dev/BeyondDFT_scripts/src/scripts/read_last_it.py", line 4, in <module>
    import h5._h5py as h5
  File "/mnt/home/cyeh/Projects/nda/nda.build/build_tensor/deps/h5/python/h5/__init__.py", line 21, in <module>
    from .archive import HDFArchive, HDFArchiveGroup, HDFArchiveInert
  File "/mnt/home/cyeh/Projects/nda/nda.build/build_tensor/deps/h5/python/h5/archive.py", line 18, in <module>
    from .archive_basic_layer import HDFArchiveGroupBasicLayer
  File "/mnt/home/cyeh/Projects/nda/nda.build/build_tensor/deps/h5/python/h5/archive_basic_layer.py", line 18, in <module>
    from . import _h5py as h5
ImportError: cannot import name '_h5py' from partially initialized module 'h5' (most likely due to a circular import) (/mnt/home/cyeh/Projects/nda/nda.build/build_tensor/deps/h5/python/h5/__init__.py)

Steps to Reproduce

The issue happens on Rusty with Python 3.10.10. The issue does not happen with Python 3.9.16. I do not see dependence on any other loaded module.

Versions

#define NDA_VERSION "1.1.0"
#define NDA_VERSION_MAJOR "1"
#define NDA_VERSION_MINOR "1"
#define NDA_VERSION_PATCH "0"
#define NDA_GIT_HASH "af14bd53403a55a977e9345da4c2867d72cfc31c"

Wentzell commented 1 month ago

Thank you @pavel-po for reporting this, and sorry for the delayed response. Are you still seeing this issue?

pavel-po commented 1 month ago

After trying to reproduce it with a fresh build, I noticed that it is sensitive to the version of python with which it has been built. As long as the version of python is the same as the one that was used to compile nda, everything works. If the python version used for running is different from the one used to build it, the issue appears. I suspect this has to do with how the *.so library is named at the compilation step: _h5py.cpython-39-x86_64-linux-gnu.so - works for python 3.9.16 _h5py.cpython-310-x86_64-linux-gnu.so - works for python 3.10.10

Wentzell commented 1 month ago

Thank you @pavel-po for the additional information. Running with a different Python Version that the one you used to compile the python bindings can indeed lead to the problem you observe. We are considering if we can catch this with a runtime check.

Wentzell commented 1 month ago

This runtime-check was now added to Cpp2Py, c.f. https://github.com/TRIQS/cpp2py/commit/7e539d4938ec919369d6284a4f63f6a40274dca2

TRIQS / nda