Stack smashing detected

[gerritholl]

When I try to run pyhdf.SD.SD(...).datasets(), or (...).select(15).get() in Python 3.4, I get Stack smashing detected. I can open the same data file fine using the traditional pyhdf in Python 2.7. I'm using Python-hdf4 0.9.

See below.

$ ipython3 --pylab
Python 3.4.1 (default, Jun 18 2014, 10:57:14) 
Type "copyright", "credits" or "license" for more information.

IPython 2.2.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
Using matplotlib backend: agg
Python are go!

In [1]: import pyhdf.SD

In [2]: sd = pyhdf.SD.SD("/data/01/gholl/data/Eureka_hdf/groundbased_ftir.ch4_utoronto001_eureka_20080301t102850z_20080920t144053z_001.hdf")

In [3]: sd.datasets()
*** stack smashing detected ***: /export/data/home/gholl/venv/gerrit/bin/python3.4 terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x348b302697]
/lib64/libc.so.6(__fortify_fail+0x0)[0x348b302660]
/export/data/home/gholl/venv/gerrit/lib/python3.4/site-packages/pyhdf/_hdfext.cpython-34m.so(+0x237ed)[0x7f6bc41767ed]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x89c2)[0x348c319592]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x82e2)[0x348c318eb2]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x82e2)[0x348c318eb2]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCode+0x3b)[0x348c31a84b]
/usr/lib64/libpython3.4m.so.1.0[0x348c30d757]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x89c2)[0x348c319592]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x82e2)[0x348c318eb2]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x787c)[0x348c31844c]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x787c)[0x348c31844c]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x787c)[0x348c31844c]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x787c)[0x348c31844c]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x82e2)[0x348c318eb2]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0[0x348c28a739]
/usr/lib64/libpython3.4m.so.1.0(PyObject_Call+0x96)[0x348c262936]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x6920)[0x348c3174f0]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalFrameEx+0x787c)[0x348c31844c]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCodeEx+0x8ce)[0x348c31a77e]
/usr/lib64/libpython3.4m.so.1.0(PyEval_EvalCode+0x3b)[0x348c31a84b]
/usr/lib64/libpython3.4m.so.1.0[0x348c337891]
/usr/lib64/libpython3.4m.so.1.0(PyRun_FileExFlags+0x115)[0x348c3379f5]
/usr/lib64/libpython3.4m.so.1.0(PyRun_SimpleFileExFlags+0x104)[0x348c33a404]
/usr/lib64/libpython3.4m.so.1.0(Py_Main+0xba0)[0x348c3502d0]
/export/data/home/gholl/venv/gerrit/bin/python3.4(main+0x165)[0x400aa5]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x348b21ed5d]
/export/data/home/gholl/venv/gerrit/bin/python3.4[0x400879]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:11 65185244                           /export/data/home/gholl/venv/gerrit/bin/python3.4
00600000-00602000 rw-p 00000000 08:11 65185244                           /export/data/home/gholl/venv/gerrit/bin/python3.4
0138e000-0204d000 rw-p 00000000 00:00 0                                  [heap]
30cf600000-30cf61d000 r-xp 00000000 08:03 3408209                        /lib64/libselinux.so.1
30cf61d000-30cf81c000 ---p 0001d000 08:03 3408209                        /lib64/libselinux.so.1
30cf81c000-30cf81d000 r--p 0001c000 08:03 3408209                        /lib64/libselinux.so.1
30cf81d000-30cf81e000 rw-p 0001d000 08:03 3408209                        /lib64/libselinux.so.1
30cf81e000-30cf81f000 rw-p 00000000 00:00 0 
30d0200000-30d0207000 r-xp 00000000 08:03 3408665                        /lib64/libcrypt-2.12.so
30d0207000-30d0407000 ---p 00007000 08:03 3408665                        /lib64/libcrypt-2.12.so
30d0407000-30d0408000 r--p 00007000 08:03 3408665                        /lib64/libcrypt-2.12.so
30d0408000-30d0409000 rw-p 00008000 08:03 3408665                        /lib64/libcrypt-2.12.so
30d0409000-30d0437000 rw-p 00000000 00:00 0 
30d0600000-30d0673000 r-xp 00000000 08:03 3408664                        /lib64/libfreebl3.so
30d0673000-30d0872000 ---p 00073000 08:03 3408664                        /lib64/libfreebl3.so
30d0872000-30d0874000 r--p 00072000 08:03 3408664                        /lib64/libfreebl3.so
30d0874000-30d0875000 rw-p 00074000 08:03 3408664                        /lib64/libfreebl3.so
30d0875000-30d0879000 rw-p 00000000 00:00 0 
30d0a00000-30d0a03000 r-xp 00000000 08:03 3409678                        /lib64/libcom_err.so.2.1
30d0a03000-30d0c02000 ---p 00003000 08:03 3409678                        /lib64/libcom_err.so.2.1
30d0c02000-30d0c03000 r--p 00002000 08:03 3409678                        /lib64/libcom_err.so.2.1
30d0c03000-30d0c04000 rw-p 00003000 08:03 3409678                        /lib64/libcom_err.so.2.1
30d0e00000-30d0e25000 r-xp 00000000 08:03 2627026                        /usr/lib64/libnssutil3.so
30d0e25000-30d1025000 ---p 00025000 08:03 2627026                        /usr/lib64/libnssutil3.so
30d1025000-30d102b000 r--p 00025000 08:03 2627026                        /usr/lib64/libnssutil3.so
30d102b000-30d102c000 rw-p 0002b000 08:03 2627026                        /usr/lib64/libnssutil3.so
30d1200000-30d1220000 r-xp 00000000 08:03 2629024                        /usr/lib64/liblzma.so.0.0.0
30d1220000-30d1420000 ---p 00020000 08:03 2629024                        /usr/lib64/liblzma.so.0.0.0
30d1420000-30d1421000 rw-p 00020000 08:03 2629024                        /usr/lib64/liblzma.so.0.0.0
30d1a00000-30d1a0a000 r-xp 00000000 08:03 3408983                        /lib64/libkrb5support.so.0.1
30d1a0a000-30d1c09000 ---p 0000a000 08:03 3408983                        /lib64/libkrb5support.so.0.1
30d1c09000-30d1c0a000 r--p 00009000 08:03 3408983                        /lib64/libkrb5support.so.0.1
30d1c0a000-30d1c0b000 rw-p 0000a000 08:03 3408983                        /lib64/libkrb5support.so.0.1
30d1e00000-30d1e02000 r-xp 00000000 08:03 3408020                        /lib64/libkeyutils.so.1.3
30d1e02000-30d2001000 ---p 00002000 08:03 3408020                        /lib64/libkeyutils.so.1.3
30d2001000-30d2002000 r--p 00001000 08:03 3408020                        /lib64/libkeyutils.so.1.3
30d2002000-30d2003000 rw-p 00002000 08:03 3408020                        /lib64/libkeyutils.so.1.3
30d2600000-30d2629000 r-xp 00000000 08:03 3408984                        /lib64/libk5crypto.so.3.1
30d2629000-30d2829000 ---p 00029000 08:03 3408984                        /lib64/libk5crypto.so.3.1
30d2829000-30d282a000 r--p 00029000 08:03 3408984                        /lib64/libk5crypto.so.3.1
30d282a000-30d282b000 rw-p 0002a000 08:03 3408984                        /lib64/libk5crypto.so.3.1
30d282b000-30d282c000 rw-p 00000000 00:00 0 
30d2a00000-30d2adb000 r-xp 00000000 08:03 3409679                        /lib64/libkrb5.so.3.3
30d2adb000-30d2cda000 ---p 000db000 08:03 3409679                        /lib64/libkrb5.so.3.3
30d2cda000-30d2ce4000 r--p 000da000 08:03 3409679                        /lib64/libkrb5.so.3.3
30d2ce4000-30d2ce6000 rw-p 000e4000 08:03 3409679                        /lib64/libkrb5.so.3.3
30d2e00000-30d2e41000 r-xp 00000000 08:03 3409680                        /lib64/libgssapi_krb5.so.2.2
30d2e41000-30d3041000 ---p 00041000 08:03 3409680                        /lib64/libgssapi_krb5.so.2.2
30d3041000-30d3042000 r--p 00041000 08:03 3409680                        /lib64/libgssapi_krb5.so.2.2
30d3042000-30d3044000 rw-p 00042000 08:03 3409680                        /lib64/libgssapi_krb5.so.2.2
30d4200000-30d4219000 r-xp 00000000 08:03 2627480                        /usr/lib64/libsasl2.so.2.0.23
30d4219000-30d4418000 ---p 00019000 08:03 2627480                        /usr/lib64/libsasl2.so.2.0.23
30d4418000-30d4419000 r--p 00018000 08:03 2627480                        /usr/lib64/libsasl2.so.2.0.23
30d4419000-30d441a000 rw-p 00019000 08:03 2627480                        /usr/lib64/libsasl2.so.2.0.23
348ae00000-348ae20000 r-xp 00000000 08:03 3408925                        /lib64/ld-2.12.so
348b01f000-348b020000 r--p 0001f000 08:03 3408925                        /lib64/ld-2.12.so
348b020000-348b021000 rw-p 00020000 08:03 3408925                        /lib64/ld-2.12.so
348b021000-348b022000 rw-p 00000000 00:00 0 
348b200000-348b38a000 r-xp 00000000 08:03 3408951                        /lib64/libc-2.12.so
348b38a000-348b58a000 ---p 0018a000 08:03 3408951                        /lib64/libc-2.12.soAborted

[gerritholl]

Using the debugger, I found out that it is triggered by:

In [4]: sd = pyhdf.SD.SD("/data/01/gholl/data/Eureka_hdf/groundbased_ftir.ch4_utoronto001_eureka_2
0080301t102850z_20080920t144053z_001.hdf")

In [5]: sds = sd.select(15)

In [6]: ipdb.run("sds.info()")
> <string>(1)<module>()

ipdb> break /export/data/home/gholl/venv/gerrit/lib/python3.4/site-packages/pyhdf/SD.py:2142
Breakpoint 1 at /export/data/home/gholl/venv/gerrit/lib/python3.4/site-packages/pyhdf/SD.py:2142
ipdb> cont
> /export/data/home/gholl/venv/gerrit/lib/python3.4/site-packages/pyhdf/SD.py(2142)info()
   2141         status, sds_name, rank, data_type, n_attrs = \
1> 2142                 _C.SDgetinfo(self._id, buf)
   2143         _checkErr('info', status, "cannot execute")
ipdb> step
*** stack smashing detected ***: /export/data/home/gholl/venv/gerrit/bin/python3.4 terminated
======= Backtrace: =========
(etc.)

[fhs]

My guess is that the variable name is too long and SDgetinfo is running out of space to store the name. Do you know the variable name and how long it is? We have a limit of 256 characters.

If the variable name length is not the issue, I'll need the HDF file to investigate further. You'll have to host it somewhere so I can download it.

[fhs]

On second thought, variable name length is probably not issue since you say it works under python 2.7.

[gerritholl]

Hi,

On 26 November 2014 at 13:06, Fazlul Shahriar notifications@github.com wrote:

On second thought, variable name length is probably not issue since you say it works under python 2.7.

The variable name is CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR_UNCERTAINTY.SYSTEMATIC.COVARIANCE

I put the file in question at

https://dl.dropboxusercontent.com/u/4650900/testfile.pdf

(I will leave it there only temporarily)

Gerrit.

[gerritholl]

On 26 November 2014 at 13:23, Gerrit Holl gerrit.holl@gmail.com wrote:

Hi,

On 26 November 2014 at 13:06, Fazlul Shahriar notifications@github.com wrote:

On second thought, variable name length is probably not issue since you say it works under python 2.7.

The variable name is CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR_UNCERTAINTY.SYSTEMATIC.COVARIANCE

I put the file in question at

https://dl.dropboxusercontent.com/u/4650900/testfile.pdf

Sorry, that should be

https://dl.dropboxusercontent.com/u/4650900/testfile.hdf

(I will leave it there only temporarily)

Gerrit.

[fhs]

I just pushed out the fix, but the fix was already there in 0.9 release except I forgot to regenerate the swig wrapper. You can generate the wrapper by running:

swig -python hdfext.i

and then install the package again.

[gerritholl]

On 26 November 2014 at 17:18, Fazlul Shahriar notifications@github.com wrote:

I just pushed out the fix, but the fix was already there in 0.9 release except I forgot to regenerate the swig wrapper. You can generate the wrapper by running:

swig -python hdfext.i

and then install the package again.

Thanks for the rapid fix, that's great!

Gerrit.