search

CDAT / cdms

8 stars 10 forks source link

Nasty segfault with erroneous netcdf-4 data #444

Open durack1 opened 2 years ago

durack1 commented 2 years ago

Describe the bug I am able to reproduce a segfault when attempting to open a poorly formed netcdf-4 file. I would expect cdms to throw an OSError or similar error, rather than segfaulting

To Reproduce Steps to reproduce the behavior: The below is a grab of an ipython console output:

In [5]: getGlobalAtts(f)

getGlobalAtts: badFile: /p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc
Error: NetCDF: HDF error

ncopen: filename "/p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc": NetCDF: HDF error
Out[5]: 
['/p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc',
 OSError('NetCDF: HDF error')]

In [6]: getGlobalAtts(f)

Output from spyder call 'get_namespace_view':
*** Error in `~/anaconda3/envs/cdms315spy515cart020/bin/python': free(): invalid pointer: 0x00007fd568424bf0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7fd570980329]
~anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5MM_xfree+0xf)[0x7fd4f9222dff]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x1aed3a)[0x7fd4f9251d3a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x1aeeb2)[0x7fd4f9251eb2]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5O_msg_reset+0x74)[0x7fd4f9252ca4]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__link_release_table+0x58)[0x7fd4f91db368]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__dense_iterate+0x14f)[0x7fd4f91d1ecf]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__obj_iterate+0xd2)[0x7fd4f91e2b32]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G_iterate+0x148)[0x7fd4f91d9b28]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x16f5ab)[0x7fd4f92125ab]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5Literate+0xca)[0x7fd4f9213f4a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(+0x945de)[0x7fd5022b85de]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(+0x976ba)[0x7fd5022bb6ba]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(NC_open+0x238)[0x7fd502254618]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(nc_open+0x15)[0x7fd5022547b5]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(ncopen+0x24)[0x7fd502273b14]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0x7689)[0x7fd5023b3689]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0x931a)[0x7fd5023b531a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0xe67e)[0x7fd5023ba67e]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x15fd78)[0x558c65f90d78]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x316)[0x558c65f776a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x534c)[0x558c66015f2c]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1ac3c7)[0x558c65fdd3c7]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x3c2)[0x558c65f77752]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x4e0c)[0x558c660159ec]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x336)[0x558c65fd1366]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyObject_Call+0x106)[0x558c65f963a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x316)[0x558c65f776a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x534c)[0x558c66015f2c]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x139900)[0x558c65f6a900]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x336)[0x558c65fd1366]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalCodeWithName+0x47)[0x558c6604f667]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyEval_EvalCodeEx+0x39)[0x558c6604f6a9]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyEval_EvalCode+0x1b)[0x558c6604f6cb]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x2930ad)[0x558c660c40ad]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x19972e)[0x558c65fca72e]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x5179)[0x558c66015d59]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x5179)[0x558c66015d59]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x19a579)[0x558c65fcb579]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x603)[0x558c660111e3]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x19a)[0x558c65fd11ca]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]

Restarting kernel...

Expected behavior I would expect and OSError to be thrown, and no segfault

Screenshots or traceback If applicable, add screenshots/traceback to help explain your problem.

Desktop (please complete the following information):

Environment Information

`conda info`

``` (cdms315spy515cart020) bash-4.2$ conda info active environment : cdms315spy515cart020 active env location : ~/anaconda3/envs/cdms315spy515cart020 shell level : 3 user config file : ~/.condarc populated config files : conda version : 4.12.0 conda-build version : 3.21.4 python version : 3.8.8.final.0 virtual packages : __linux=3.10.0=0 __glibc=2.17=0 __unix=0=0 __archspec=1=x86_64 base environment : ~/anaconda3 (writable) conda av data dir : ~/anaconda3/etc/conda conda av metadata url : None channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : ~/anaconda3/pkgs ~/.conda/pkgs envs directories : ~/anaconda3/envs ~/.conda/envs platform : linux-64 user-agent : conda/4.12.0 requests/2.27.1 CPython/3.8.8 Linux/3.10.0-1160.42.2.el7.x86_64 rhel/7.9 glibc/2.17 UID:GID : 40336:1026 netrc file : None offline mode : False ```

`conda config --show-sources`

``` ```

`conda list --show-channel-urls`

``` ```

Additional context Add any other context about the problem here.