search

hdmf-dev / hdmf-zarr

Zarr I/O backend for HDMF
https://hdmf-zarr.readthedocs.io/
Other
7 stars 6 forks source link

[Bug]: numpy 2.0 causes pynwb ZarrIO tutorial to fail #213

Open pauladkisson opened 2 months ago

pauladkisson commented 2 months ago

What happened?

Tried to follow pynwb ZarrIO tutorial as a base case for working with ZarrIO, but it fails with the latest hdmf_zarr from github.

Tried downgrading numpy from 2.0 --> 1.26.4 and the error was resolved.

Steps to Reproduce

# Create the NWBFile. Substitute your NWBFile generation here.
nwbfile = NWBFile(
    session_description="my first synthetic recording",
    identifier="EXAMPLE_ID",
    session_start_time=datetime.now(tzlocal()),
    session_id="LONELYMTN",
)

data_with_zarr_data_io = ZarrDataIO(
    data=np.random.randn(100, 100),
    chunks=(10, 10),
    fillvalue=0,
    compressor=Blosc(cname='zstd', clevel=3, shuffle=Blosc.SHUFFLE)
)
nwbfile.add_acquisition(
    TimeSeries(
        name="synthetic_timeseries",
        data=data_with_zarr_data_io,
        unit="m",
        rate=10e3,
    )
)

path = "zarr_tutorial.nwb.zarr"
with NWBZarrIO(path=path, mode="w") as io:
    io.write(nwbfile)

Traceback

Traceback (most recent call last):
  File "/Users/pauladkisson/Documents/CatalystNeuro/HDMF/hdmf-zarr/temp_test.py", line 5, in <module>
    from pynwb import NWBFile, TimeSeries
  File "/opt/anaconda3/envs/hdmf_zarr_env/lib/python3.9/site-packages/pynwb/__init__.py", line 8, in <module>
    import h5py
  File "/opt/anaconda3/envs/hdmf_zarr_env/lib/python3.9/site-packages/h5py/__init__.py", line 45, in <module>
    from ._conv import register_converters as _register_converters, \
  File "h5py/_conv.pyx", line 1, in init h5py._conv
  File "h5py/h5r.pyx", line 1, in init h5py.h5r
  File "h5py/h5p.pyx", line 1, in init h5py.h5p
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Operating System

macOS

Python Executable

Conda

Python Version

3.9

Package Versions

For the environment, I followed installation instructions here: https://hdmf-zarr.readthedocs.io/en/latest/installation.html

Code of Conduct

mavaylon1 commented 2 months ago

Hmmm that is weird. Thank you for the issue ticket. I will take a look today to see what is going on!

I will let you of a timeline then.

mavaylon1 commented 2 months ago

Hmmm the gallery tests are passing on the scheduler.

mavaylon1 commented 2 months ago

Ah the tests are using the numpy v1. I'm not sure why though as the requirements files state numpy 2.

mavaylon1 commented 2 months ago

I also tried your example with numpy 2.0 and it works. Can you try creating a clean environment and both trying it out again and show me your pip list

pauladkisson commented 2 months ago

I was already using a clean env, but I tried it out again just in case (same error).

Here is pip list output:

Package                       Version             Editable project location
----------------------------- ------------------- ----------------------------------------------------------
airium                        0.2.6
alabaster                     0.7.16
annotated-types               0.7.0
antlr4-python3-runtime        4.9.3
appdirs                       1.4.4
arrow                         1.3.0
asciitree                     0.3.3
attrs                         24.2.0
babel                         2.16.0
bcp47                         0.1.0
beautifulsoup4                4.12.3
bioregistry                   0.11.14
black                         24.3.0
cachetools                    5.5.0
cattrs                        23.2.3
certifi                       2024.7.4
CFGraph                       0.2.1
cfgv                          3.4.0
chardet                       5.2.0
charset-normalizer            3.3.2
class_resolver                0.4.3
click                         8.1.7
codespell                     2.2.6
colorama                      0.4.6
contourpy                     1.2.1
coverage                      7.3.2
curies                        0.7.10
cycler                        0.12.1
daff                          1.3.46
defusedxml                    0.7.1
Deprecated                    1.2.14
deprecation                   2.1.0
distlib                       0.3.8
docutils                      0.20.1
et-xmlfile                    1.1.0
eutils                        0.6.0
exceptiongroup                1.2.2
fasteners                     0.19
fastobo                       0.12.3
filelock                      3.15.4
fonttools                     4.53.1
fqdn                          1.5.1
funowl                        0.2.3
google                        3.0.0
google-api-core               2.19.1
google-api-python-client      2.142.0
google-auth                   2.34.0
google-auth-httplib2          0.2.0
google-auth-oauthlib          1.2.1
googleapis-common-protos      1.63.2
graphviz                      0.20.3
greenlet                      3.0.3
gspread                       6.1.2
gspread-formatting            1.2.0
h5py                          3.11.0
hbreader                      0.9.1
hdf5plugin                    4.3.0
hdmf                          3.14.3
hdmf_zarr                     0.8.1.dev6+g82c993c /Users/pauladkisson/Documents/CatalystNeuro/HDMF/hdmf-zarr
httplib2                      0.22.0
identify                      2.6.0
idna                          3.7
ijson                         3.3.0
imagesize                     1.4.1
importlib_metadata            8.4.0
importlib_resources           6.4.3
iniconfig                     2.0.0
isodate                       0.6.1
isoduration                   20.11.0
Jinja2                        3.1.4
json-flattener                0.1.9
jsonasobj                     1.3.1
jsonasobj2                    1.0.4
jsonlines                     4.0.0
jsonpatch                     1.33
jsonpath-ng                   1.6.1
jsonpointer                   3.0.0
jsonschema                    4.22.0
jsonschema-specifications     2023.12.1
kgcl-rdflib                   0.5.0
kgcl_schema                   0.6.9
kiwisolver                    1.4.5
lark                          1.2.2
linkml                        1.7.10
linkml-dataops                0.1.0
linkml-renderer               0.3.1
linkml-runtime                1.7.7
lxml                          5.3.0
MarkupSafe                    2.1.5
matplotlib                    3.9.2
mkl-fft                       1.3.8
mkl-random                    1.2.4
mkl-service                   2.4.0
more-click                    0.1.2
mypy-extensions               1.0.0
ndex2                         3.9.0
networkx                      3.2.1
nodeenv                       1.9.1
numcodecs                     0.12.1
numpy                         2.0.0
oaklib                        0.6.10
oauthlib                      3.2.2
ols-client                    0.1.4
ontodev-cogs                  0.3.3
ontoportal-client             0.0.4
openpyxl                      3.1.5
packaging                     24.1
pandas                        2.2.2
pansql                        0.0.1
parse                         1.20.2
pathspec                      0.12.1
pillow                        10.4.0
pip                           24.2
platformdirs                  4.2.2
pluggy                        1.5.0
ply                           3.11
pre-commit                    3.5.0
prefixcommons                 0.1.12
prefixmaps                    0.2.5
pronto                        2.5.7
proto-plus                    1.24.0
protobuf                      5.27.3
pyasn1                        0.6.0
pyasn1_modules                0.4.0
pydantic                      2.8.2
pydantic_core                 2.20.1
Pygments                      2.18.0
PyJSG                         0.11.10
pynwb                         2.5.0
pyparsing                     3.1.2
pyproject-api                 1.7.1
PyShEx                        0.8.1
PyShExC                       0.9.1
pysolr                        3.9.0
pystow                        0.5.4
pytest                        7.4.3
pytest-cov                    4.1.0
pytest-logging                2015.11.4
python-dateutil               2.8.2
PyTrie                        0.4.0
pytz                          2024.1
PyYAML                        6.0.2
ratelimit                     2.2.1
rdflib                        7.0.0
rdflib-jsonld                 0.6.1
rdflib-shim                   1.0.3
referencing                   0.35.1
requests                      2.32.3
requests-cache                1.2.1
requests-oauthlib             2.0.0
requests-toolbelt             1.0.0
rfc3339-validator             0.1.4
rfc3987                       1.3.8
rpds-py                       0.20.0
rsa                           4.9
ruamel.yaml                   0.18.2
ruamel.yaml.clib              0.2.8
ruff                          0.1.3
schemasheets                  0.2.1
scipy                         1.13.1
semsql                        0.3.3
setuptools                    72.1.0
ShExJSG                       0.8.2
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
soupsieve                     2.6
sparqlslurper                 0.5.1
SPARQLWrapper                 2.0.0
Sphinx                        7.4.7
sphinx-copybutton             0.5.2
sphinx-gallery                0.17.1
sphinx-rtd-theme              2.0.0
sphinxcontrib-applehelp       2.0.0
sphinxcontrib-devhelp         2.0.0
sphinxcontrib-htmlhelp        2.1.0
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          2.0.0
sphinxcontrib-serializinghtml 2.0.0
SQLAlchemy                    2.0.32
SQLAlchemy-Utils              0.38.3
sssom                         0.4.11
sssom-schema                  0.15.2
tabulate                      0.9.0
tenacity                      8.5.0
termcolor                     2.4.0
threadpoolctl                 3.2.0
tomli                         2.0.1
tox                           4.11.3
tqdm                          4.66.4
types-python-dateutil         2.9.0.20240821
typing_extensions             4.12.2
tzdata                        2024.1
uri-template                  1.3.0
uritemplate                   4.1.1
url-normalize                 1.4.3
urllib3                       2.2.2
validators                    0.33.0
virtualenv                    20.26.3
watchdog                      4.0.2
webcolors                     24.8.0
wheel                         0.43.0
wrapt                         1.16.0
zarr                          2.16.1
zipp                          3.20.0
mavaylon1 commented 2 months ago

@pauladkisson could you do the following:

  1. activate the env
  2. touch environment.yml
  3. conda env export > environment.yml
  4. paste the content of that file here
mavaylon1 commented 2 months ago

With this I can recreate the exact env on my end to try it out.

pauladkisson commented 2 months ago 
name: hdmf_zarr_env2
channels:
  - defaults
dependencies:
  - blas=1.0=mkl
  - c-ares=1.19.1=h6c40b1e_0
  - ca-certificates=2024.7.2=hecd8cb5_0
  - h5py=3.11.0=py39h8cd6c6c_0
  - hdf5=1.12.1=ha01d115_3
  - intel-openmp=2023.1.0=ha357a0b_43548
  - krb5=1.20.1=h428f121_1
  - libcurl=8.7.1=hf20ceda_0
  - libcxx=14.0.6=h9765a3e_0
  - libedit=3.1.20230828=h6c40b1e_0
  - libev=4.33=h9ed2024_1
  - libffi=3.4.4=hecd8cb5_1
  - libgfortran=5.0.0=11_3_0_hecd8cb5_28
  - libgfortran5=11.3.0=h9dfd629_28
  - libnghttp2=1.57.0=h9beae6a_0
  - libssh2=1.11.0=hf20ceda_0
  - llvm-openmp=14.0.6=h0dcd299_0
  - mkl=2023.1.0=h8e150cf_43560
  - mkl-service=2.4.0=py39h6c40b1e_1
  - mkl_fft=1.3.8=py39h6c40b1e_0
  - mkl_random=1.2.4=py39ha357a0b_0
  - ncurses=6.4=hcec6c5f_0
  - openssl=3.0.14=h46256e1_0
  - pip=24.2=py39hecd8cb5_0
  - python=3.9.19=h5ee71fb_1
  - readline=8.2=hca72f7f_0
  - setuptools=72.1.0=py39hecd8cb5_0
  - sqlite=3.45.3=h6c40b1e_0
  - tbb=2021.8.0=ha357a0b_0
  - tk=8.6.14=h4d00af3_0
  - wheel=0.43.0=py39hecd8cb5_0
  - xz=5.4.6=h6c40b1e_1
  - zlib=1.2.13=h4b97444_1
  - pip:
      - airium==0.2.6
      - alabaster==0.7.16
      - annotated-types==0.7.0
      - antlr4-python3-runtime==4.9.3
      - appdirs==1.4.4
      - arrow==1.3.0
      - asciitree==0.3.3
      - attrs==24.2.0
      - babel==2.16.0
      - bcp47==0.1.0
      - beautifulsoup4==4.12.3
      - bioregistry==0.11.14
      - black==24.3.0
      - cachetools==5.5.0
      - cattrs==23.2.3
      - certifi==2024.7.4
      - cfgraph==0.2.1
      - cfgv==3.4.0
      - chardet==5.2.0
      - charset-normalizer==3.3.2
      - class-resolver==0.4.3
      - click==8.1.7
      - codespell==2.2.6
      - colorama==0.4.6
      - contourpy==1.2.1
      - coverage==7.3.2
      - curies==0.7.10
      - cycler==0.12.1
      - daff==1.3.46
      - defusedxml==0.7.1
      - deprecated==1.2.14
      - deprecation==2.1.0
      - distlib==0.3.8
      - docutils==0.20.1
      - et-xmlfile==1.1.0
      - eutils==0.6.0
      - exceptiongroup==1.2.2
      - fasteners==0.19
      - fastobo==0.12.3
      - filelock==3.15.4
      - fonttools==4.53.1
      - fqdn==1.5.1
      - funowl==0.2.3
      - google==3.0.0
      - google-api-core==2.19.1
      - google-api-python-client==2.142.0
      - google-auth==2.34.0
      - google-auth-httplib2==0.2.0
      - google-auth-oauthlib==1.2.1
      - googleapis-common-protos==1.63.2
      - greenlet==3.0.3
      - gspread==6.1.2
      - gspread-formatting==1.2.0
      - hbreader==0.9.1
      - hdf5plugin==4.3.0
      - hdmf==3.14.3
      - hdmf-zarr==0.8.1.dev6+g82c993c
      - httplib2==0.22.0
      - identify==2.6.0
      - idna==3.7
      - ijson==3.3.0
      - imagesize==1.4.1
      - importlib-metadata==8.4.0
      - importlib-resources==6.4.3
      - iniconfig==2.0.0
      - isodate==0.6.1
      - isoduration==20.11.0
      - jinja2==3.1.4
      - json-flattener==0.1.9
      - jsonasobj==1.3.1
      - jsonasobj2==1.0.4
      - jsonlines==4.0.0
      - jsonpatch==1.33
      - jsonpath-ng==1.6.1
      - jsonpointer==3.0.0
      - jsonschema==4.22.0
      - jsonschema-specifications==2023.12.1
      - kgcl-rdflib==0.5.0
      - kgcl-schema==0.6.9
      - kiwisolver==1.4.5
      - lark==1.2.2
      - linkml==1.7.10
      - linkml-dataops==0.1.0
      - linkml-renderer==0.3.1
      - linkml-runtime==1.7.7
      - lxml==5.3.0
      - markupsafe==2.1.5
      - matplotlib==3.9.2
      - more-click==0.1.2
      - mypy-extensions==1.0.0
      - ndex2==3.9.0
      - networkx==3.2.1
      - nodeenv==1.9.1
      - numcodecs==0.12.1
      - numpy==2.0.0
      - oaklib==0.6.10
      - oauthlib==3.2.2
      - ols-client==0.1.4
      - ontodev-cogs==0.3.3
      - ontoportal-client==0.0.4
      - openpyxl==3.1.5
      - packaging==24.1
      - pandas==2.2.2
      - pansql==0.0.1
      - parse==1.20.2
      - pathspec==0.12.1
      - pillow==10.4.0
      - platformdirs==4.2.2
      - pluggy==1.5.0
      - ply==3.11
      - pre-commit==3.5.0
      - prefixcommons==0.1.12
      - prefixmaps==0.2.5
      - pronto==2.5.7
      - proto-plus==1.24.0
      - protobuf==5.27.3
      - pyasn1==0.6.0
      - pyasn1-modules==0.4.0
      - pydantic==2.8.2
      - pydantic-core==2.20.1
      - pygments==2.18.0
      - pyjsg==0.11.10
      - pynwb==2.5.0
      - pyparsing==3.1.2
      - pyproject-api==1.7.1
      - pyshex==0.8.1
      - pyshexc==0.9.1
      - pysolr==3.9.0
      - pystow==0.5.4
      - pytest==7.4.3
      - pytest-cov==4.1.0
      - pytest-logging==2015.11.4
      - python-dateutil==2.8.2
      - python-graphviz==0.20.3
      - pytrie==0.4.0
      - pytz==2024.1
      - pyyaml==6.0.2
      - ratelimit==2.2.1
      - rdflib==7.0.0
      - rdflib-jsonld==0.6.1
      - rdflib-shim==1.0.3
      - referencing==0.35.1
      - requests==2.32.3
      - requests-cache==1.2.1
      - requests-oauthlib==2.0.0
      - requests-toolbelt==1.0.0
      - rfc3339-validator==0.1.4
      - rfc3987==1.3.8
      - rpds-py==0.20.0
      - rsa==4.9
      - ruamel-yaml==0.18.2
      - ruamel-yaml-clib==0.2.8
      - ruff==0.1.3
      - schemasheets==0.2.1
      - scipy==1.13.1
      - semsql==0.3.3
      - shexjsg==0.8.2
      - six==1.16.0
      - snowballstemmer==2.2.0
      - sortedcontainers==2.4.0
      - soupsieve==2.6
      - sparqlslurper==0.5.1
      - sparqlwrapper==2.0.0
      - sphinx==7.4.7
      - sphinx-copybutton==0.5.2
      - sphinx-gallery==0.17.1
      - sphinx-rtd-theme==2.0.0
      - sphinxcontrib-applehelp==2.0.0
      - sphinxcontrib-devhelp==2.0.0
      - sphinxcontrib-htmlhelp==2.1.0
      - sphinxcontrib-jquery==4.1
      - sphinxcontrib-jsmath==1.0.1
      - sphinxcontrib-qthelp==2.0.0
      - sphinxcontrib-serializinghtml==2.0.0
      - sqlalchemy==2.0.32
      - sqlalchemy-utils==0.38.3
      - sssom==0.4.11
      - sssom-schema==0.15.2
      - tabulate==0.9.0
      - tenacity==8.5.0
      - termcolor==2.4.0
      - threadpoolctl==3.2.0
      - tomli==2.0.1
      - tox==4.11.3
      - tqdm==4.66.4
      - types-python-dateutil==2.9.0.20240821
      - typing-extensions==4.12.2
      - tzdata==2024.1
      - uri-template==1.3.0
      - uritemplate==4.1.1
      - url-normalize==1.4.3
      - urllib3==2.2.2
      - validators==0.33.0
      - virtualenv==20.26.3
      - watchdog==4.0.2
      - webcolors==24.8.0
      - wrapt==1.16.0
      - zarr==2.16.1
      - zipp==3.20.0
prefix: /opt/anaconda3/envs/hdmf_zarr_env2
mavaylon1 commented 2 months ago

Hey @pauladkisson here are a few updates and a rough timeline:

  1. I see the error comes from just importing hdf5. I installed hdf5 (your version) and swapped between numpy version and it still works.
  2. PyNWB restricts numpy to below 2, but that isn't affecting the workflow and even when I do it locally it works fine.
  3. When I try creating your environment, it fails. I see you have packages like linkml in there. In that env, did you just run "pip install hdmf-zarr". I'm curious to how you have linkml because that is an optional package in hdmf.

Since downgrading numpy is essentially working, I will take a look again next week.

pauladkisson commented 2 months ago

Thanks for looking into this.

In that env, did you just run "pip install hdmf-zarr"?

I was following the dev instructions here, so I installed all the requirements as well:

conda create --name hdmf-zarr-test python=3.9
conda activate hdmf-zarr-test
conda install h5py

git clone --recurse-submodules https://github.com/hdmf-dev/hdmf.git
cd hdmf
pip install -r requirements.txt -r requirements-dev.txt -r requirements-doc.txt -r requirements-opt.txt
pip install -e .
cd ..

git clone https://github.com/hdmf-dev/hdmf-zarr.git
cd hdmf-zarr
pip install -r requirements.txt -r requirements-dev.txt -r requirements-doc.txt
pip install -e .
rly commented 1 month ago

Pynwb 2.8.2 was released today which adds support for numpy 2. Gallery tests are now failing: https://github.com/hdmf-dev/hdmf-zarr/actions/runs/10786265512. The two may be related. @mavaylon1 can you take a look?

rly commented 2 weeks ago

FYI Gallery tests are now passing

mavaylon1 commented 2 weeks ago

Can you try again now? It should be working now due to updates from both pynwb and zarr packages. @pauladkisson

pauladkisson commented 1 week ago

Still failing with new env and latest changes from github.

pauladkisson commented 1 week ago

Here's the environment for reference:

name: hdmf_zarr_env
channels:
  - defaults
dependencies:
  - blas=1.0=mkl
  - c-ares=1.19.1=h6c40b1e_0
  - ca-certificates=2024.9.24=hecd8cb5_0
  - h5py=3.11.0=py39h8cd6c6c_0
  - hdf5=1.12.1=ha01d115_3
  - intel-openmp=2023.1.0=ha357a0b_43548
  - krb5=1.20.1=h428f121_1
  - libcurl=8.9.1=h3a17b82_0
  - libcxx=14.0.6=h9765a3e_0
  - libedit=3.1.20230828=h6c40b1e_0
  - libev=4.33=h9ed2024_1
  - libffi=3.4.4=hecd8cb5_1
  - libgfortran=5.0.0=11_3_0_hecd8cb5_28
  - libgfortran5=11.3.0=h9dfd629_28
  - libnghttp2=1.57.0=h9beae6a_0
  - libssh2=1.11.0=hf20ceda_0
  - llvm-openmp=14.0.6=h0dcd299_0
  - mkl=2023.1.0=h8e150cf_43560
  - mkl-service=2.4.0=py39h6c40b1e_1
  - mkl_fft=1.3.8=py39h6c40b1e_0
  - mkl_random=1.2.4=py39ha357a0b_0
  - ncurses=6.4=hcec6c5f_0
  - openssl=3.0.15=h46256e1_0
  - pip=24.2=py39hecd8cb5_0
  - python=3.9.20=hce00570_1
  - readline=8.2=hca72f7f_0
  - setuptools=75.1.0=py39hecd8cb5_0
  - sqlite=3.45.3=h6c40b1e_0
  - tbb=2021.8.0=ha357a0b_0
  - tk=8.6.14=h4d00af3_0
  - wheel=0.44.0=py39hecd8cb5_0
  - xz=5.4.6=h6c40b1e_1
  - zlib=1.2.13=h4b97444_1
  - pip:
      - airium==0.2.6
      - alabaster==0.7.16
      - annotated-types==0.7.0
      - antlr4-python3-runtime==4.9.3
      - appdirs==1.4.4
      - arrow==1.3.0
      - asciitree==0.3.3
      - attrs==24.2.0
      - babel==2.16.0
      - bcp47==0.1.0
      - beautifulsoup4==4.12.3
      - bioregistry==0.11.18
      - black==24.3.0
      - cachetools==5.5.0
      - cattrs==24.1.2
      - certifi==2024.8.30
      - cfgraph==0.2.1
      - cfgv==3.4.0
      - chardet==5.2.0
      - charset-normalizer==3.4.0
      - class-resolver==0.5.2
      - click==8.1.7
      - codespell==2.2.6
      - colorama==0.4.6
      - contourpy==1.3.0
      - coverage==7.3.2
      - curies==0.8.0
      - cycler==0.12.1
      - daff==1.3.46
      - defusedxml==0.7.1
      - deprecated==1.2.14
      - deprecation==2.1.0
      - distlib==0.3.9
      - docutils==0.21.2
      - et-xmlfile==1.1.0
      - eutils==0.6.0
      - exceptiongroup==1.2.2
      - fasteners==0.19
      - fastobo==0.12.3
      - filelock==3.16.1
      - fonttools==4.54.1
      - fqdn==1.5.1
      - funowl==0.2.3
      - google==3.0.0
      - google-api-core==2.21.0
      - google-api-python-client==2.149.0
      - google-auth==2.35.0
      - google-auth-httplib2==0.2.0
      - google-auth-oauthlib==1.2.1
      - googleapis-common-protos==1.65.0
      - greenlet==3.1.1
      - gspread==6.1.4
      - gspread-formatting==1.2.0
      - hbreader==0.9.1
      - hdf5plugin==4.3.0
      - hdmf==3.14.5
      - hdmf-zarr==0.9.1.dev1+g46bdb77
      - html5lib-modern==1.2
      - httplib2==0.22.0
      - identify==2.6.1
      - idna==3.10
      - ijson==3.3.0
      - imagesize==1.4.1
      - importlib-metadata==8.5.0
      - importlib-resources==6.4.5
      - iniconfig==2.0.0
      - isodate==0.7.2
      - isoduration==20.11.0
      - jinja2==3.1.4
      - json-flattener==0.1.9
      - jsonasobj==1.3.1
      - jsonasobj2==1.0.4
      - jsonlines==4.0.0
      - jsonpatch==1.33
      - jsonpath-ng==1.7.0
      - jsonpointer==3.0.0
      - jsonschema==4.22.0
      - jsonschema-specifications==2024.10.1
      - kgcl-rdflib==0.5.0
      - kgcl-schema==0.6.9
      - kiwisolver==1.4.7
      - lark==1.2.2
      - linkml==1.7.10
      - linkml-dataops==0.1.0
      - linkml-renderer==0.3.1
      - linkml-runtime==1.7.7
      - lxml==5.3.0
      - markupsafe==3.0.2
      - matplotlib==3.9.2
      - more-click==0.1.2
      - mypy-extensions==1.0.0
      - ndex2==3.9.0
      - networkx==3.2.1
      - nodeenv==1.9.1
      - numcodecs==0.12.1
      - numpy==2.0.0
      - oaklib==0.6.10
      - oauthlib==3.2.2
      - ols-client==0.1.4
      - ontodev-cogs==0.3.3
      - ontoportal-client==0.0.4
      - openpyxl==3.1.5
      - packaging==24.1
      - pandas==2.2.2
      - pansql==0.0.1
      - parse==1.20.2
      - pathspec==0.12.1
      - pillow==11.0.0
      - platformdirs==4.3.6
      - pluggy==1.5.0
      - ply==3.11
      - pre-commit==3.5.0
      - prefixcommons==0.1.12
      - prefixmaps==0.2.6
      - pronto==2.5.8
      - proto-plus==1.24.0
      - protobuf==5.28.2
      - pyasn1==0.6.1
      - pyasn1-modules==0.4.1
      - pydantic==2.9.2
      - pydantic-core==2.23.4
      - pygments==2.18.0
      - pyjsg==0.11.10
      - pynwb==2.5.0
      - pyparsing==3.2.0
      - pyproject-api==1.8.0
      - pyshex==0.8.1
      - pyshexc==0.9.1
      - pysolr==3.10.0
      - pystow==0.5.6
      - pytest==7.4.3
      - pytest-cov==4.1.0
      - pytest-logging==2015.11.4
      - python-dateutil==2.8.2
      - python-graphviz==0.20.3
      - pytrie==0.4.0
      - pytz==2024.2
      - pyyaml==6.0.2
      - ratelimit==2.2.1
      - rdflib==7.1.0
      - rdflib-jsonld==0.6.1
      - rdflib-shim==1.0.3
      - referencing==0.35.1
      - requests==2.32.3
      - requests-cache==1.2.1
      - requests-oauthlib==2.0.0
      - requests-toolbelt==1.0.0
      - rfc3339-validator==0.1.4
      - rfc3987==1.3.8
      - rpds-py==0.20.0
      - rsa==4.9
      - ruamel-yaml==0.18.2
      - ruamel-yaml-clib==0.2.12
      - ruff==0.1.3
      - schemasheets==0.2.1
      - scipy==1.13.1
      - semsql==0.3.3
      - shexjsg==0.8.2
      - six==1.16.0
      - snowballstemmer==2.2.0
      - sortedcontainers==2.4.0
      - soupsieve==2.6
      - sparqlslurper==0.5.1
      - sparqlwrapper==2.0.0
      - sphinx==7.4.7
      - sphinx-copybutton==0.5.2
      - sphinx-gallery==0.18.0
      - sphinx-rtd-theme==3.0.1
      - sphinxcontrib-applehelp==2.0.0
      - sphinxcontrib-devhelp==2.0.0
      - sphinxcontrib-htmlhelp==2.1.0
      - sphinxcontrib-jquery==4.1
      - sphinxcontrib-jsmath==1.0.1
      - sphinxcontrib-qthelp==2.0.0
      - sphinxcontrib-serializinghtml==2.0.0
      - sqlalchemy==2.0.36
      - sqlalchemy-utils==0.38.3
      - sssom==0.4.11
      - sssom-schema==0.15.2
      - tabulate==0.9.0
      - tenacity==8.5.0
      - termcolor==2.5.0
      - threadpoolctl==3.2.0
      - tomli==2.0.2
      - tox==4.11.3
      - tqdm==4.66.4
      - types-python-dateutil==2.9.0.20241003
      - typing-extensions==4.12.2
      - tzdata==2024.2
      - uri-template==1.3.0
      - uritemplate==4.1.1
      - url-normalize==1.4.3
      - urllib3==2.2.3
      - validators==0.34.0
      - virtualenv==20.27.0
      - watchdog==5.0.3
      - webcolors==24.8.0
      - wrapt==1.16.0
      - zarr==2.16.1
      - zipp==3.20.2
prefix: /opt/anaconda3/envs/hdmf_zarr_env
mavaylon1 commented 1 week ago

Here's the environment for reference:


name: hdmf_zarr_env

channels:

  - defaults

dependencies:

  - blas=1.0=mkl

  - c-ares=1.19.1=h6c40b1e_0

  - ca-certificates=2024.9.24=hecd8cb5_0

  - h5py=3.11.0=py39h8cd6c6c_0

  - hdf5=1.12.1=ha01d115_3

  - intel-openmp=2023.1.0=ha357a0b_43548

  - krb5=1.20.1=h428f121_1

  - libcurl=8.9.1=h3a17b82_0

  - libcxx=14.0.6=h9765a3e_0

  - libedit=3.1.20230828=h6c40b1e_0

  - libev=4.33=h9ed2024_1

  - libffi=3.4.4=hecd8cb5_1

  - libgfortran=5.0.0=11_3_0_hecd8cb5_28

  - libgfortran5=11.3.0=h9dfd629_28

  - libnghttp2=1.57.0=h9beae6a_0

  - libssh2=1.11.0=hf20ceda_0

  - llvm-openmp=14.0.6=h0dcd299_0

  - mkl=2023.1.0=h8e150cf_43560

  - mkl-service=2.4.0=py39h6c40b1e_1

  - mkl_fft=1.3.8=py39h6c40b1e_0

  - mkl_random=1.2.4=py39ha357a0b_0

  - ncurses=6.4=hcec6c5f_0

  - openssl=3.0.15=h46256e1_0

  - pip=24.2=py39hecd8cb5_0

  - python=3.9.20=hce00570_1

  - readline=8.2=hca72f7f_0

  - setuptools=75.1.0=py39hecd8cb5_0

  - sqlite=3.45.3=h6c40b1e_0

  - tbb=2021.8.0=ha357a0b_0

  - tk=8.6.14=h4d00af3_0

  - wheel=0.44.0=py39hecd8cb5_0

  - xz=5.4.6=h6c40b1e_1

  - zlib=1.2.13=h4b97444_1

  - pip:

      - airium==0.2.6

      - alabaster==0.7.16

      - annotated-types==0.7.0

      - antlr4-python3-runtime==4.9.3

      - appdirs==1.4.4

      - arrow==1.3.0

      - asciitree==0.3.3

      - attrs==24.2.0

      - babel==2.16.0

      - bcp47==0.1.0

      - beautifulsoup4==4.12.3

      - bioregistry==0.11.18

      - black==24.3.0

      - cachetools==5.5.0

      - cattrs==24.1.2

      - certifi==2024.8.30

      - cfgraph==0.2.1

      - cfgv==3.4.0

      - chardet==5.2.0

      - charset-normalizer==3.4.0

      - class-resolver==0.5.2

      - click==8.1.7

      - codespell==2.2.6

      - colorama==0.4.6

      - contourpy==1.3.0

      - coverage==7.3.2

      - curies==0.8.0

      - cycler==0.12.1

      - daff==1.3.46

      - defusedxml==0.7.1

      - deprecated==1.2.14

      - deprecation==2.1.0

      - distlib==0.3.9

      - docutils==0.21.2

      - et-xmlfile==1.1.0

      - eutils==0.6.0

      - exceptiongroup==1.2.2

      - fasteners==0.19

      - fastobo==0.12.3

      - filelock==3.16.1

      - fonttools==4.54.1

      - fqdn==1.5.1

      - funowl==0.2.3

      - google==3.0.0

      - google-api-core==2.21.0

      - google-api-python-client==2.149.0

      - google-auth==2.35.0

      - google-auth-httplib2==0.2.0

      - google-auth-oauthlib==1.2.1

      - googleapis-common-protos==1.65.0

      - greenlet==3.1.1

      - gspread==6.1.4

      - gspread-formatting==1.2.0

      - hbreader==0.9.1

      - hdf5plugin==4.3.0

      - hdmf==3.14.5

      - hdmf-zarr==0.9.1.dev1+g46bdb77

      - html5lib-modern==1.2

      - httplib2==0.22.0

      - identify==2.6.1

      - idna==3.10

      - ijson==3.3.0

      - imagesize==1.4.1

      - importlib-metadata==8.5.0

      - importlib-resources==6.4.5

      - iniconfig==2.0.0

      - isodate==0.7.2

      - isoduration==20.11.0

      - jinja2==3.1.4

      - json-flattener==0.1.9

      - jsonasobj==1.3.1

      - jsonasobj2==1.0.4

      - jsonlines==4.0.0

      - jsonpatch==1.33

      - jsonpath-ng==1.7.0

      - jsonpointer==3.0.0

      - jsonschema==4.22.0

      - jsonschema-specifications==2024.10.1

      - kgcl-rdflib==0.5.0

      - kgcl-schema==0.6.9

      - kiwisolver==1.4.7

      - lark==1.2.2

      - linkml==1.7.10

      - linkml-dataops==0.1.0

      - linkml-renderer==0.3.1

      - linkml-runtime==1.7.7

      - lxml==5.3.0

      - markupsafe==3.0.2

      - matplotlib==3.9.2

      - more-click==0.1.2

      - mypy-extensions==1.0.0

      - ndex2==3.9.0

      - networkx==3.2.1

      - nodeenv==1.9.1

      - numcodecs==0.12.1

      - numpy==2.0.0

      - oaklib==0.6.10

      - oauthlib==3.2.2

      - ols-client==0.1.4

      - ontodev-cogs==0.3.3

      - ontoportal-client==0.0.4

      - openpyxl==3.1.5

      - packaging==24.1

      - pandas==2.2.2

      - pansql==0.0.1

      - parse==1.20.2

      - pathspec==0.12.1

      - pillow==11.0.0

      - platformdirs==4.3.6

      - pluggy==1.5.0

      - ply==3.11

      - pre-commit==3.5.0

      - prefixcommons==0.1.12

      - prefixmaps==0.2.6

      - pronto==2.5.8

      - proto-plus==1.24.0

      - protobuf==5.28.2

      - pyasn1==0.6.1

      - pyasn1-modules==0.4.1

      - pydantic==2.9.2

      - pydantic-core==2.23.4

      - pygments==2.18.0

      - pyjsg==0.11.10

      - pynwb==2.5.0

      - pyparsing==3.2.0

      - pyproject-api==1.8.0

      - pyshex==0.8.1

      - pyshexc==0.9.1

      - pysolr==3.10.0

      - pystow==0.5.6

      - pytest==7.4.3

      - pytest-cov==4.1.0

      - pytest-logging==2015.11.4

      - python-dateutil==2.8.2

      - python-graphviz==0.20.3

      - pytrie==0.4.0

      - pytz==2024.2

      - pyyaml==6.0.2

      - ratelimit==2.2.1

      - rdflib==7.1.0

      - rdflib-jsonld==0.6.1

      - rdflib-shim==1.0.3

      - referencing==0.35.1

      - requests==2.32.3

      - requests-cache==1.2.1

      - requests-oauthlib==2.0.0

      - requests-toolbelt==1.0.0

      - rfc3339-validator==0.1.4

      - rfc3987==1.3.8

      - rpds-py==0.20.0

      - rsa==4.9

      - ruamel-yaml==0.18.2

      - ruamel-yaml-clib==0.2.12

      - ruff==0.1.3

      - schemasheets==0.2.1

      - scipy==1.13.1

      - semsql==0.3.3

      - shexjsg==0.8.2

      - six==1.16.0

      - snowballstemmer==2.2.0

      - sortedcontainers==2.4.0

      - soupsieve==2.6

      - sparqlslurper==0.5.1

      - sparqlwrapper==2.0.0

      - sphinx==7.4.7

      - sphinx-copybutton==0.5.2

      - sphinx-gallery==0.18.0

      - sphinx-rtd-theme==3.0.1

      - sphinxcontrib-applehelp==2.0.0

      - sphinxcontrib-devhelp==2.0.0

      - sphinxcontrib-htmlhelp==2.1.0

      - sphinxcontrib-jquery==4.1

      - sphinxcontrib-jsmath==1.0.1

      - sphinxcontrib-qthelp==2.0.0

      - sphinxcontrib-serializinghtml==2.0.0

      - sqlalchemy==2.0.36

      - sqlalchemy-utils==0.38.3

      - sssom==0.4.11

      - sssom-schema==0.15.2

      - tabulate==0.9.0

      - tenacity==8.5.0

      - termcolor==2.5.0

      - threadpoolctl==3.2.0

      - tomli==2.0.2

      - tox==4.11.3

      - tqdm==4.66.4

      - types-python-dateutil==2.9.0.20241003

      - typing-extensions==4.12.2

      - tzdata==2024.2

      - uri-template==1.3.0

      - uritemplate==4.1.1

      - url-normalize==1.4.3

      - urllib3==2.2.3

      - validators==0.34.0

      - virtualenv==20.27.0

      - watchdog==5.0.3

      - webcolors==24.8.0

      - wrapt==1.16.0

      - zarr==2.16.1

      - zipp==3.20.2

prefix: /opt/anaconda3/envs/hdmf_zarr_env

I will look into this