DennisHeimbigner
commented
1 year ago First, what configure options (or CMake options) did you use? Second, what value did you use for HDF5_PLUGIN_PATH?
Closed minomicetto closed 1 year ago
DennisHeimbigner
commented
1 year ago First, what configure options (or CMake options) did you use? Second, what value did you use for HDF5_PLUGIN_PATH?
minomicetto
commented
1 year ago I built the library by appending the option --with-plugin-dir as follows:
./configure --prefix=/path/to/netcdf-4.9.2 --enable-s3 --with-plugin-dir=/path/to/netcdf-4.9.2/lib/plugins LDFLAGS="-L/path/to/aws-sdk-cpp/lib64 -laws-cpp-sdk-s3" CPPFLAGS=-I/path/to/aws-sdk-cpp/include CC=mpicc CPP=mpicxx
This resulted that in /path/to/netcdf-4.9.2/lib/plugins I can find the related libraries:
ls /path/to/netcdf-4.9.2/lib/plugins libnch5blosc.la libnch5bzip2.so.0 libnch5fletcher32.la libnch5shuffle.so.0 libnch5zstd.la libnczhdf5filters.so.0 libnch5blosc.so libnch5bzip2.so.0.0.0 libnch5fletcher32.so libnch5shuffle.so.0.0.0 libnch5zstd.so libnczhdf5filters.so.0.0.0 libnch5blosc.so.0 libnch5deflate.la libnch5fletcher32.so.0 libnch5szip.la libnch5zstd.so.0 libnczstdfilters.la libnch5blosc.so.0.0.0 libnch5deflate.so libnch5fletcher32.so.0.0.0 libnch5szip.so libnch5zstd.so.0.0.0 libnczstdfilters.so libnch5bzip2.la libnch5deflate.so.0 libnch5shuffle.la libnch5szip.so.0 libnczhdf5filters.la libnczstdfilters.so.0 libnch5bzip2.so libnch5deflate.so.0.0.0 libnch5shuffle.so libnch5szip.so.0.0.0 libnczhdf5filters.so libnczstdfilters.so.0.0.0
Then, I set HDF5_PLUGIN_PATH to /path/to/netcdf-4.9.2/lib/plugins as follows:
export HDF5_PLUGIN_PATH=/path/to/netcdf-4.9.2/lib/plugin
and now all the tests passed. I thank you.
minomicetto
commented
1 year ago Now, I am testing my application using the NetCDF library and... the same error is raised. However, in this case the problem is not due to HDF5_PLUGIN_PATH as the env variable is correctly set and most of above plugins are loaded correctly (all the plugins named lib__nch5*). In particular, by appending a simple print of dlerror() at https://github.com/Unidata/netcdf-c/blob/main/libncpoco/cp_unix.c#L137 I obtained that
dlopen(): /path/to/netcdf-4.9.2/lib/plugins/libnczhdf5filters.so.0: undefined symbol: nc_inq_var_endian dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczhdf5filters.so: undefined symbol: nc_inq_var_endian dlopen(): /path/to/netcdf-4.9.2/lib/plugins/libnczstdfilters.so.0: undefined symbol: nc_inq_type dlopen(): /path/to/netcdf-4.9.2/lib/plugins/libnczstdfilters.so: undefined symbol: nc_inq_type dlopen(): /path/to/netcdf-4.9.2/lib/plugins/libnczhdf5filters.so.0.0.0: undefined symbol: nc_inq_var_endian dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so.0.0.0: undefined symbol: nc_inq_type
The problem occurs even if LD_LIBRARY_PATH includes /path/to/netcdf-4.9.2/lib/plugins
I have obtained a similar problem even over another architecture
Linux n179 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
but, in this case, already the execution of dopen() results in error as follows:
symbol lookup error: /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so.0: undefined symbol: nc_inq_var
Could you help me to solve this issue?
WardF
commented
1 year ago I'll take a look, I'm curious if the LD_LIBRARY_PATH also contains the path to libnetcdf; it is complaining about undefined symbols that should be found in libnetcdf.
DennisHeimbigner
commented
1 year ago Not all filters need to use libnetcdf, so check the Makefile to see if this one has it.
minomicetto
commented
1 year ago @WardF: my LD_LIBRARY_PATH already includes libnetcdf as I am testing an application that uses it.
LD_LIBRARY_PATH=/path/to/hdf5-1.14.1-2/lib:/path/to/netcdf-4.9.2/lib:/path/to/aws-sdk-cpp/lib64
To this I have appended even the reference to /path/to/netcdf-4.9.2/lib/plugins, however without resulting in success.
@DennisHeimbigner: I am going to check the Makefile of the plugins and to refer you, but I have found no solution so far. Let me say if you suggest something to do, please.
minomicetto
commented
1 year ago I have solved simply by forcing the library to link itself. Maybe, this affects only the intended plugins, as the reflexive link does not impact the building phase (while the library does not exist yet). In pratice, I have used the following configuration:
./configure --prefix=/path/to/netcdf-4.9.2 --enable-s3 LDFLAGS="-L/path/to/aws-sdk-cpp/lib64 -laws-cpp-sdk-s3 -L/path/to/netcdf-4.9.2/lib -lnetcdf" CPPFLAGS=-I/path/to/aws-sdk-cpp/include CC=mpicc CPP=mpicxx
I think there is a more smart approach to solve the problem but, in the meantime, we can close this issue.
DennisHeimbigner
commented
1 year ago Try adding directory containing libnetcdf to LD_LIBRARY_PATH
minomicetto
commented
1 year ago @DennisHeimbigner: as I replied before, LD_LIBRARY_PATH has always included the path to libnetcdf in my tests; in fact, my application exploits it and they are regularly linked:
$ ldd /path/to/my/application linux-vdso.so.1 (0x0000ffffa4d41000) libnetcdf.so.19 => /path/to/libnetcdf.so.19 (0x0000ffffa4a00000) libltdl.so.7 => /lib64/libltdl.so.7 (0x0000ffffa4ce3000) libm.so.6 => /lib64/libm.so.6 (0x0000ffffa4c42000) libc.so.6 => /lib64/libc.so.6 (0x0000ffffa4852000) [...]
The problem is the link between each ncz plugin and libnetcdf: even if the LD_LIBRARY_PATH includes libnetcdf (useless during the building), the plugins are built without an explicit reference to it. In fact, the ldd tool applied to a filter does not report the connection:
$ ldd /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so linux-vdso.so.1 (0x0000ffff879d1000) libc.so.6 => /lib64/libc.so.6 (0x0000ffff877c5000) /lib/ld-linux-aarch64.so.1 (0x0000ffff87994000)
I think that connection should rightly be established on loading the plugin, but this does not happend in my case. Instead, adding the reference to libnetcdf to LDFLAGS, the filter includes an explicit connection even before loading it (and many other links, in effect):
$ ldd /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so linux-vdso.so.1 (0x0000ffff98cbd000) libnetcdf.so.19 => /path/to/libnetcdf.so.19 (0x0000ffff98a00000) libc.so.6 => /lib64/libc.so.6 (0x0000ffff98852000) /lib/ld-linux-aarch64.so.1 (0x0000ffff98c80000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000ffff98600000) libaws-cpp-sdk-s3.so => /path/to/libaws-cpp-sdk-s3.so (0x0000ffff98200000) libhdf5_hl.so.310 => /path/to/libhdf5_hl.so.310 (0x0000ffff985ce000) libhdf5.so.310 => /path/to/libhdf5.so.310 (0x0000ffff97c00000) libm.so.6 => /lib64/libm.so.6 (0x0000ffff9852d000) [...]
Then, I have no trouble to run the plugins. Clearly, a smarter solution to link libnetcdf (only) to each ncz plugin, using default configuration, would be prefarable. I thank you.
DennisHeimbigner
commented
1 year ago ok, please do ldd /path/to/netcdf-4.9.2/lib/plugins/lib__nczhdf5filters.so.0 to see where it thinks libnetcdf is located.
# Platform specification Architecture: aarch64 NetCDF C Library: 4.9.2, but the same problem is raised even for 4.9.0 Operating System: Linux CentOS 9 Stream - Linux localhost.localdomain 5.14.0-340.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Jul 17 12:31:16 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-redhat-linux/11/lto-wrapper Target: aarch64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-gnu-indirect-function --build=aarch64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1 Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 11.4.1 20230605 (Red Hat 11.4.1-2) (GCC)
mpicc -v mpicc for MPICH version 4.1.1 Using built-in specs. Reading specs from /usr/lib/rpm/redhat/redhat-hardened-cc1 Reading specs from /usr/lib/rpm/redhat/redhat-annobin-cc1 COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-redhat-linux/11/lto-wrapper Target: aarch64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-gnu-indirect-function --build=aarch64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1 Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 11.4.1 20230605 (Red Hat 11.4.1-2) (GCC)
mpicxx -v mpicxx for MPICH version 4.1.1 Using built-in specs. Reading specs from /usr/lib/rpm/redhat/redhat-hardened-cc1 Reading specs from /usr/lib/rpm/redhat/redhat-annobin-cc1 COLLECT_GCC=g++ COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-redhat-linux/11/lto-wrapper Target: aarch64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-gnu-indirect-function --build=aarch64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1 Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 11.4.1 20230605 (Red Hat 11.4.1-2) (GCC)
The steps performed to install and to access a Zarr dataset via the library are listed below.
# Download and install s3 library git clone https://github.com/aws/aws-sdk-cpp.git cd aws-sdk-cpp git submodule update --init --recursive srcdir=$(pwd) mkdir build cd build cmake $srcdir -DBUILD_ONLY=s3 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/path/to/aws-sdk-cpp make -j2 -k make install
# Download and install NetCDF library wget https://downloads.unidata.ucar.edu/netcdf-c/4.9.2/netcdf-c-4.9.2.zip unzip netcdf-c-4.9.2.zip cd netcdf-c-4.9.2 ./configure --prefix=/path/to/netcdf-4.9.2 --enable-s3 LDFLAGS="-L/path/to/aws-sdk-cpp/lib64 -laws-cpp-sdk-s3" CPPFLAGS=-I/path/to/aws-sdk-cpp/include CC=mpicc CPP=mpicxx make -j2 -k make install
# Download the input Zarr data cd /data/zarr wget -r -H -N --cut-dirs=4 --include-directories="/v1/" "https://swiftbrowser.dkrz.de/public/dkrz_0b2a0dcc-1430-4a8a-9f25-a6cb8924d92b/CMCC_CMIP/CMIP6.CMIP.CMCC.CMCC-CM2-HR4.historical.r1i1p1f1.day.pr.gn.v20200904/?show_all"
The tests are listed below:
# Tests export LD_LIBRARY_PATH=/path/to/netcdf-4.9.2/lib:/path/to/aws-sdk-cpp/lib64:$LD_LIBRARY_PATH
## Test 1: passed /path/to/netcdf-4.9.2/bin/ncdump -h s3://storage.googleapis.com/cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/highresSST-present/r1i1p1f1/Amon/rsds/gn/v20170706/#mode=zarr netcdf #mode\=zarr { dimensions: lat = 192 ; bnds = 2 ; lon = 288 ; time = 804 ; variables: double lat(lat) ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:long_name = "latitude" ; lat:standard_name = "latitude" ; lat:units = "degrees_north" ; double lat_bnds(lat, bnds) ; double lon(lon) ; lon:axis = "X" ; lon:bounds = "lon_bnds" ; lon:long_name = "longitude" ; lon:standard_name = "longitude" ; lon:units = "degrees_east" ; double lon_bnds(lon, bnds) ; float rsds(time, lat, lon) ; rsds:cell_measures = "area: areacella" ; rsds:cell_methods = "area: time: mean" ; rsds:comment = "surface solar irradiance for UV calculations" ; rsds:history = "2017-07-02T15:44:13Z altered by CMOR: Converted units from \' W m-2\' to \'W m-2\'." ; rsds:long_name = "Surface Downwelling Shortwave Radiation" ; rsds:original_units = "W m-2" ; rsds:standard_name = "surface_downwelling_shortwave_flux_in_air" ; rsds:units = "W m-2" ; int64 time(time) ; time:axis = "T" ; time:bounds = "time_bnds" ; time:calendar = "noleap" ; time:long_name = "time" ; time:standard_name = "time" ; time:units = "hours since 1948-01-16 12:00:00.000000" ; double time_bnds(time, bnds) ; time_bnds:calendar = "365_day" ; time_bnds:units = "days since 1948-01-01" ;
// global attributes: :CDI = "Climate Data Interface version 1.8.0 (http://mpimet.mpg.de/cdi)" ; :CDO = "Climate Data Operators version 1.8.0 (http://mpimet.mpg.de/cdo)" ; :Conventions = "CF-1.7 CMIP-6.0" ; :activity_id = "HighResMIP" ; :cmor_version = "3.2.3" ; :coordinates = "lon_bnds time_bnds lat_bnds" ; :creation_date = "2017-07-02T15:44:13Z" ; :data_specs_version = "01.00.23" ; :end_year = "2014" ; :experiment = "forced atmosphere experiment for 1950-2014" ; :experiment_id = "highresSST-present" ; :external_variables = "areacella" ; :forcing_index = 1 ; :frequency = "mon" ; :further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.highresSST-present.none.r1i1p1f1" ; :grid = "native atmosphere regular grid (192x288 latxlon)" ; :grid_label = "gn" ; :history = "2017-07-02T15:44:13Z CMOR rewrote data to be consistent with CF standards and CMIP6 requirements.; 2019-04-23T15:50:59Z DataSpecsVersionAdd, FurtherInfoUrlToHttps" ; :initialization_index = 1 ; :institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ; :institution_id = "CMCC" ; :license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ; :min_number_yrs_per_sim = "65" ; :mip_era = "CMIP6" ; :nominal_resolution = "1x1 degree" ; :parent_activity_id = "no parent" ; :parent_experiment_id = "no parent" ; :parent_mip_era = "no parent" ; :physics_index = 1 ; :product = "model-output" ; :realization_index = 1 ; :realm = "atmos" ; :source = "CMCC-CM2-HR4 (2016): aerosol: prescribed MACv2-SP atmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa) atmosChem: none land: CLM4.5 (SP mode) landIce: none ocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m) ocnBgchem: none seaIce: CICE4.0" ; :source_id = "CMCC-CM2-HR4" ; :source_type = "AGCM" ; :start_year = "1950" ; :sub_experiment = "none" ; :sub_experiment_id = "none" ; :table_id = "Amon" ; :table_info = "Creation Date:(22 April 2017) MD5:d3f61e4124f67845cd50ff647ba0c673" ; :tier = "1" ; :title = "CMCC-CM2-HR4 model output prepared for CMIP6" ; :tracking_id = "hdl:21.14100/6bc46076-c6c8-4f5b-acf0-d79baab75d7e" ; :variable_id = "rsds" ; :variant_label = "r1i1p1f1" ; :status = "2019-11-16;created;by nhn2@columbia.edu" ; }
## Test 2: failed /path/to/netcdf-4.9.2/bin/ncdump -hs s3://storage.googleapis.com/cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/highresSST-present/r1i1p1f1/Amon/rsds/gn/v20170706/#mode=zarr netcdf #mode\=zarr { dimensions: lat = 192 ; bnds = 2 ; lon = 288 ; time = 804 ; variables: double lat(lat) ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:long_name = "latitude" ; lat:standard_name = "latitude" ; lat:units = "degrees_north" ; lat:_Storage = "chunked" ; lat:_ChunkSizes = 192 ; NetCDF: Filter error: undefined filter encountered Location: file ncdump.c; fcn pr_att_specials line 1060
## Test 3: failed /path/to/netcdf-4.9.2/bin/ncdump -v lat s3://storage.googleapis.com/cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/highresSST-present/r1i1p1f1/Amon/rsds/gn/v20170706/#mode=zarr netcdf #mode\=zarr { dimensions: lat = 192 ; bnds = 2 ; lon = 288 ; time = 804 ; variables: double lat(lat) ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:long_name = "latitude" ; lat:standard_name = "latitude" ; lat:units = "degrees_north" ; double lat_bnds(lat, bnds) ; double lon(lon) ; lon:axis = "X" ; lon:bounds = "lon_bnds" ; lon:long_name = "longitude" ; lon:standard_name = "longitude" ; lon:units = "degrees_east" ; double lon_bnds(lon, bnds) ; float rsds(time, lat, lon) ; rsds:cell_measures = "area: areacella" ; rsds:cell_methods = "area: time: mean" ; rsds:comment = "surface solar irradiance for UV calculations" ; rsds:history = "2017-07-02T15:44:13Z altered by CMOR: Converted units from \' W m-2\' to \'W m-2\'." ; rsds:long_name = "Surface Downwelling Shortwave Radiation" ; rsds:original_units = "W m-2" ; rsds:standard_name = "surface_downwelling_shortwave_flux_in_air" ; rsds:units = "W m-2" ; int64 time(time) ; time:axis = "T" ; time:bounds = "time_bnds" ; time:calendar = "noleap" ; time:long_name = "time" ; time:standard_name = "time" ; time:units = "hours since 1948-01-16 12:00:00.000000" ; double time_bnds(time, bnds) ; time_bnds:calendar = "365_day" ; time_bnds:units = "days since 1948-01-01" ;
// global attributes: :CDI = "Climate Data Interface version 1.8.0 (http://mpimet.mpg.de/cdi)" ; :CDO = "Climate Data Operators version 1.8.0 (http://mpimet.mpg.de/cdo)" ; :Conventions = "CF-1.7 CMIP-6.0" ; :activity_id = "HighResMIP" ; :cmor_version = "3.2.3" ; :coordinates = "lon_bnds time_bnds lat_bnds" ; :creation_date = "2017-07-02T15:44:13Z" ; :data_specs_version = "01.00.23" ; :end_year = "2014" ; :experiment = "forced atmosphere experiment for 1950-2014" ; :experiment_id = "highresSST-present" ; :external_variables = "areacella" ; :forcing_index = 1 ; :frequency = "mon" ; :further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.highresSST-present.none.r1i1p1f1" ; :grid = "native atmosphere regular grid (192x288 latxlon)" ; :grid_label = "gn" ; :history = "2017-07-02T15:44:13Z CMOR rewrote data to be consistent with CF standards and CMIP6 requirements.; 2019-04-23T15:50:59Z DataSpecsVersionAdd, FurtherInfoUrlToHttps" ; :initialization_index = 1 ; :institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ; :institution_id = "CMCC" ; :license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ; :min_number_yrs_per_sim = "65" ; :mip_era = "CMIP6" ; :nominal_resolution = "1x1 degree" ; :parent_activity_id = "no parent" ; :parent_experiment_id = "no parent" ; :parent_mip_era = "no parent" ; :physics_index = 1 ; :product = "model-output" ; :realization_index = 1 ; :realm = "atmos" ; :source = "CMCC-CM2-HR4 (2016): aerosol: prescribed MACv2-SP atmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa) atmosChem: none land: CLM4.5 (SP mode) landIce: none ocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m) ocnBgchem: none seaIce: CICE4.0" ; :source_id = "CMCC-CM2-HR4" ; :source_type = "AGCM" ; :start_year = "1950" ; :sub_experiment = "none" ; :sub_experiment_id = "none" ; :table_id = "Amon" ; :table_info = "Creation Date:(22 April 2017) MD5:d3f61e4124f67845cd50ff647ba0c673" ; :tier = "1" ; :title = "CMCC-CM2-HR4 model output prepared for CMIP6" ; :tracking_id = "hdl:21.14100/6bc46076-c6c8-4f5b-acf0-d79baab75d7e" ; :variable_id = "rsds" ; :variant_label = "r1i1p1f1" ; :status = "2019-11-16;created;by nhn2@columbia.edu" ; data:
NetCDF: Filter error: undefined filter encountered Location: file vardata.c; fcn print_rows line 478
## Test 4: passed /path/to/netcdf-4.9.2/bin/ncdump -h file:///data/zarr/swift.dkrz.de#mode=zarr netcdf swift.dkrz { dimensions: lat = 192 ; bnds = 2 ; lon = 288 ; time = 60225 ; variables: double lat(lat) ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:long_name = "Latitude" ; lat:standard_name = "latitude" ; lat:units = "degrees_north" ; double lat_bnds(lat, bnds) ; double lon(lon) ; lon:axis = "X" ; lon:bounds = "lon_bnds" ; lon:long_name = "Longitude" ; lon:standard_name = "longitude" ; lon:units = "degrees_east" ; double lon_bnds(lon, bnds) ; float pr(time, lat, lon) ; pr:cell_measures = "area: areacella" ; pr:cell_methods = "area: time: mean" ; pr:comment = "includes both liquid and solid phases" ; pr:long_name = "Precipitation" ; pr:missing_value = 1.00000002004088e+20 ; pr:original_name = "PRECT" ; pr:standard_name = "precipitation_flux" ; pr:units = "kg m-2 s-1" ; int64 time(time) ; time:axis = "T" ; time:bounds = "time_bnds" ; time:calendar = "noleap" ; time:long_name = "time" ; time:standard_name = "time" ; time:units = "days since 1850-01-01 12:00:00.000000" ; double time_bnds(time, bnds) ; time_bnds:calendar = "365_day" ; time_bnds:units = "days since 1850-01-01" ;
// global attributes: :Conventions = "CF-1.7 CMIP-6.2" ; :activity_id = "CMIP" ; :branch_method = "standard" ; :branch_time_in_child = 0. ; :branch_time_in_parent = 0. ; :cmor_version = "3.5.0" ; :comment = "none" ; :contact = "T. Lovato" ; :data_specs_version = "01.00.31" ; :experiment = "all-forcing simulation of the recent past" ; :experiment_id = "historical" ; :external_variables = "areacella" ; :forcing_index = 1 ; :frequency = "day" ; :further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.historical.none.r1i1p1f1" ; :grid = "native atmosphere regular grid" ; :grid_label = "gn" ; :history = "Converted and written to swift cloud with tzis version 0.1" ; :initialization_index = 1 ; :institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ; :institution_id = "CMCC" ; :license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ; :mip_era = "CMIP6" ; :nominal_resolution = "100 km" ; :parent_activity_id = "CMIP" ; :parent_experiment_id = "piControl" ; :parent_mip_era = "CMIP6" ; :parent_source_id = "CMCC-CM2-HR4" ; :parent_time_units = "days since 1850-01-01" ; :parent_variant_label = "r1i1p1f1" ; :physics_index = 1 ; :product = "model-output" ; :realization_index = 1 ; :realm = "atmos" ; :references = "none" ; :run_variant = "1st realization" ; :source = "CMCC-CM2-HR4 (2016): \naerosol: prescribed MACv2-SP\natmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa)\natmosChem: none\nland: CLM4.5 (SP mode)\nlandIce: none\nocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m)\nocnBgchem: none\nseaIce: CICE4.0" ; :source_id = "CMCC-CM2-HR4" ; :source_type = "AOGCM" ; :sub_experiment = "none" ; :sub_experiment_id = "none" ; :table_id = "day" ; :table_info = "Creation Date:(05 February 2020) MD5:6a248fd76c55aa6d6f7b3cc6866b5faf" ; :title = "CMCC-CM2-HR4 output prepared for CMIP6" ; :tracking_id = "7af29565-027f-4d81-a74b-bee9a1235c0d" ; :variable_id = "pr" ; :variant_label = "r1i1p1f1" ; }
## Test 5: failed /path/to/netcdf-4.9.2/bin/ncdump -hs file:///data/zarr/swift.dkrz.de#mode=zarr netcdf swift.dkrz { dimensions: lat = 192 ; bnds = 2 ; lon = 288 ; time = 60225 ; variables: double lat(lat) ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:long_name = "Latitude" ; lat:standard_name = "latitude" ; lat:units = "degrees_north" ; lat:_Storage = "chunked" ; lat:_ChunkSizes = 192 ; NetCDF: Filter error: undefined filter encountered Location: file ncdump.c; fcn pr_att_specials line 1060
## Test 6: failed /path/to/netcdf-4.9.2/bin/ncdump -v lat file:///data/zarr/swift.dkrz.de#mode=zarr netcdf swift.dkrz { dimensions: lat = 192 ; bnds = 2 ; lon = 288 ; time = 60225 ; variables: double lat(lat) ; lat:axis = "Y" ; lat:bounds = "lat_bnds" ; lat:long_name = "Latitude" ; lat:standard_name = "latitude" ; lat:units = "degrees_north" ; double lat_bnds(lat, bnds) ; double lon(lon) ; lon:axis = "X" ; lon:bounds = "lon_bnds" ; lon:long_name = "Longitude" ; lon:standard_name = "longitude" ; lon:units = "degrees_east" ; double lon_bnds(lon, bnds) ; float pr(time, lat, lon) ; pr:cell_measures = "area: areacella" ; pr:cell_methods = "area: time: mean" ; pr:comment = "includes both liquid and solid phases" ; pr:long_name = "Precipitation" ; pr:missing_value = 1.00000002004088e+20 ; pr:original_name = "PRECT" ; pr:standard_name = "precipitation_flux" ; pr:units = "kg m-2 s-1" ; int64 time(time) ; time:axis = "T" ; time:bounds = "time_bnds" ; time:calendar = "noleap" ; time:long_name = "time" ; time:standard_name = "time" ; time:units = "days since 1850-01-01 12:00:00.000000" ; double time_bnds(time, bnds) ; time_bnds:calendar = "365_day" ; time_bnds:units = "days since 1850-01-01" ;
// global attributes: :Conventions = "CF-1.7 CMIP-6.2" ; :activity_id = "CMIP" ; :branch_method = "standard" ; :branch_time_in_child = 0. ; :branch_time_in_parent = 0. ; :cmor_version = "3.5.0" ; :comment = "none" ; :contact = "T. Lovato" ; :data_specs_version = "01.00.31" ; :experiment = "all-forcing simulation of the recent past" ; :experiment_id = "historical" ; :external_variables = "areacella" ; :forcing_index = 1 ; :frequency = "day" ; :further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.historical.none.r1i1p1f1" ; :grid = "native atmosphere regular grid" ; :grid_label = "gn" ; :history = "Converted and written to swift cloud with tzis version 0.1" ; :initialization_index = 1 ; :institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ; :institution_id = "CMCC" ; :license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ; :mip_era = "CMIP6" ; :nominal_resolution = "100 km" ; :parent_activity_id = "CMIP" ; :parent_experiment_id = "piControl" ; :parent_mip_era = "CMIP6" ; :parent_source_id = "CMCC-CM2-HR4" ; :parent_time_units = "days since 1850-01-01" ; :parent_variant_label = "r1i1p1f1" ; :physics_index = 1 ; :product = "model-output" ; :realization_index = 1 ; :realm = "atmos" ; :references = "none" ; :run_variant = "1st realization" ; :source = "CMCC-CM2-HR4 (2016): \naerosol: prescribed MACv2-SP\natmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa)\natmosChem: none\nland: CLM4.5 (SP mode)\nlandIce: none\nocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m)\nocnBgchem: none\nseaIce: CICE4.0" ; :source_id = "CMCC-CM2-HR4" ; :source_type = "AOGCM" ; :sub_experiment = "none" ; :sub_experiment_id = "none" ; :table_id = "day" ; :table_info = "Creation Date:(05 February 2020) MD5:6a248fd76c55aa6d6f7b3cc6866b5faf" ; :title = "CMCC-CM2-HR4 output prepared for CMIP6" ; :tracking_id = "7af29565-027f-4d81-a74b-bee9a1235c0d" ; :variable_id = "pr" ; :variant_label = "r1i1p1f1" ; data:
NetCDF: Filter error: undefined filter encountered Location: file vardata.c; fcn print_rows line 478 lat =
I think the errors raised in tests 2, 3, 5 and 6 are due to the same cause. Could you help me to solve this issue?