Segmentation fault with python3.6 with 3.0.0<=h5py<=3.1.0

[kyleaoman]

Describe the bug There is an issue apparently introduced in h5py 3.0.0 that causes a memory allocation issue leading to a segmentation fault.

To Reproduce Steps to reproduce the behaviour:

1. Working in the tests directory of swiftgalaxy (for access to toysnap.py).
2. python -c "from toysnap import *;create_toysnap();create_toyvr();from swiftgalaxy import *;vr=Velociraptor('toyvr', halo_index=0);sg=SWIFTGalaxy('toysnap.hdf5', vr);sg.gas.hydrogen_ionization_fractions.neutral" (note using a SWIFTDataset directly but naively doesn't seem to reproduce the error, but presumably would if the same mask was provided before invoking the read).
3. Segmentation fault (core dumped)

Expected behaviour Data is read cleanly and program exits.

System and python details (please complete):

• OS:

  DISTRIB_ID=LinuxMint
  DISTRIB_RELEASE=20.3
  DISTRIB_CODENAME=una
  DISTRIB_DESCRIPTION="Linux Mint 20.3 Una"
  NAME="Linux Mint"
  VERSION="20.3 (Una)"
  ID=linuxmint
  ID_LIKE=ubuntu
  PRETTY_NAME="Linux Mint 20.3"
  VERSION_ID="20.3"
  HOME_URL="https://www.linuxmint.com/"
  SUPPORT_URL="https://forums.linuxmint.com/"
  BUG_REPORT_URL="http://linuxmint-troubleshooting-guide.readthedocs.io/en/latest/"
  PRIVACY_POLICY_URL="https://www.linuxmint.com/"
  VERSION_CODENAME=una
  UBUNTU_CODENAME=focal

• Python version: Python 3.6.13 :: Anaconda, Inc.
• swiftgalaxy version, and versions of dependencies:

  astropy==4.1
  attrs==21.4.0
  cached-property==1.5.2
  certifi==2020.6.20
  Cython==0.29.28
  h5py==3.0.0
  importlib-metadata==4.8.3
  iniconfig==1.1.1
  llvmlite==0.36.0
  mpmath==1.2.1
  numba==0.53.1
  numpy==1.19.5
  packaging==21.3
  pluggy==1.0.0
  py==1.11.0
  pyparsing==3.0.8
  pytest==7.0.1
  scipy==1.5.4
  six==1.16.0
  -e git+ssh://git@github.com/SWIFTSIM/swiftgalaxy.git@249692dd06b5ab74dd733e727a23e3061ff4d677#egg=swiftgalaxy
  -e git+ssh://git@github.com/kyleaoman/swiftsimio@a157029803f5584585b123421440e85b5e72269c#egg=swiftsimio
  sympy==1.9
  tomli==1.2.3
  typing_extensions==4.1.1
  unyt==2.8.0
  velociraptor==0.15.3
  zipp==3.6.0

Additional context Some crashes complained of "corrupted chunks", errors in free() or malloc(), GC issues, and more. Crash seemed to occur around return statement of read_ranges_from_file_unchunked in swiftsimio/accelerated.py, but seems to actually be triggered somewhere in a garbage collection call while interpreting code in h5py.

[kyleaoman]

@JBorrow this came up in my tests, but smells like an upstream issue (probably in h5py but possibly in swiftsimio), thought you'd want to be aware. Or if this looks familiar, please let me know!

[kyleaoman]

The real issue is upstream in h5py. Details noted in https://github.com/SWIFTSIM/swiftsimio/issues/127

[kyleaoman]

h5py releases >=3.1.0 for python3.6 won't happen, but swiftsimio and swiftgalaxy no longer support python3.6, on any more recent python the solution is simply to update h5py to a recent version.