IGPwinter opening bug

[ThomasHaine]

• OceanSpy version:

New (16Mar23) Oceanography image on SciServer.

Description

Overwhelming error reporting when trying to open the dataset. Command never successfully returns with the opened dataset.

What I Did

import oceanspy as ospy
od = ospy.open_oceandataset.from_catalog('IGPwinter')

HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
  #000: H5A.c line 528 in H5Aopen_by_name(): can't open attribute
    major: Attribute
    minor: Can't open object
  #001: H5VLcallback.c line 1091 in H5VL_attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #002: H5VLcallback.c line 1058 in H5VL__attr_open(): attribute open failed
    major: Virtual Object Layer
    minor: Can't open object
  #003: H5VLnative_attr.c line 130 in H5VL__native_attr_open(): can't open attribute
    major: Attribute
    minor: Can't open object
  #004: H5Aint.c line 545 in H5A__open_by_name(): unable to load attribute info from object header
    major: Attribute
    minor: Unable to initialize object
  #005: H5Oattribute.c line 494 in H5O__attr_open_by_name(): can't locate attribute: '_QuantizeBitGroomNumberOfSignificantDigits'
    major: Attribute
    minor: Object not found

[Mikejmnez]

that is a very overwhelming error! I tried too and got the same issues.

To add context, oceanspy can open other files like LLC4320 and "get_started" just fine. Hard to tell what the error is, but I will take a look later tonight and / or tomorrow.

[malmans2]

Looks like the issue has been discussed here, you'll probably have to pin NetCDF: https://github.com/pydata/xarray/issues/7549

[Mikejmnez]

Thanks @malmans2! Easy fix, and I'll start by correctly labeling this issue

[Mikejmnez]

OK. I figured it out. It is not netcdf4 that needs to be pinned down, but a library libnetcdf<4.8.2 (see https://github.com/SciTools/iris/issues/5187). version 4.8.1 works fine with 'netcdf4=1.6.2' (which I think is the current version installed in the Oceanography image).

I ran a test env in Sciserver and it all works fine (IGP can open and all)

PR coming soon!

[ThomasHaine]

This issue persists with the new version. @Mikejmnez this was supposed to fix the issue, right?

[Mikejmnez]

@ThomasHaine did you install the new environment already?

[ThomasHaine]

@Mikejmnez I installed the oceanspy version from an hour ago that pins netcdf (see the version number in the previous comment). Is there something else to update?

[Mikejmnez]

Yeah. Either a new oceanography image with the corrected environment (pinned netcdf4) or, alternatively, you can open a terminal and do the following:

conda activate Oceanography
conda install netcdf4 = 1.6.1

that should install the correct version of netcdf4. You can check that that version got installed in the terminal

conda list

and check that not only netcdf4 == 1.6.1 but importantly, the libnetcdf<4.8.2. This last one is the one with the problems.

If everthing is OK, then restart the notebook and try again

[Mikejmnez]

BTW - Mitya built a test Oceanography image which I tested with and works with oceanspy. It needs to be build again to get the latest PR of oceanspy (I had to do pip install to get the latest corrections). It should be available late Wed or by early Thurs.

[ThomasHaine]

Thanks @Mikejmnez, I understand now. And it works! I'll test the new Oceanography image when it's ready (please let me know).

[malmans2]

@Mikejmnez, I haven't tested it yet, but I think the new release of libnetcdf is meant to fix the warnings: https://anaconda.org/conda-forge/libnetcdf

[Mikejmnez]

Great, I'll check this later today. Thanks @malmans2 !