segmentation fault when reading two cbf files

[chuckie82]

An attempt to read two cbf files creates segmentation fault.

[psanagpu103:02587] Process received signal [psanagpu103:02587] Signal: Segmentation fault (11) [psanagpu103:02587] Signal code: Address not mapped (1) [psanagpu103:02587] Failing at address: 0x580f758

[simongregorebner]

Would it be possible that you can provide me the two cbf files so that I can debug the cause?

[chuckie82]

Thank you for looking into this. Could you try downloading these two files: https://drive.google.com/drive/folders/0B8D6EIN_bgNVUkhpdHo5SmdVYXM?usp=sharing Please only use for debugging. I will delete the link once you've downloaded.

In [1]: import cbf In [2]: a = cbf.read('a.cbf') In [3]: b = cbf.read('b.cbf')

[simongregorebner]

@chuckie82 - Thanks a lot for the files, I downloaded them. Of course handle them with full discretion and delete them after debugging

[simongregorebner]

I tested the files with the latest code version of this package and was not able to reproduce your problems with your files on Mac OS X and Linux (SL6 64 bit) both with Python 2.7 and Python 3.5. I compiled and uploaded the latest version of the code (i.e. version 1.1.1) to anaconda central so could you please install the very latest version via conda install -c paulscherrerinstitute cbf=1.1.1 and let me know the issue still exists.

If the issues still exist, can you give me more details about the Python you are using (2.7, 3.5, numpy version, etc.) and the operating system you are on. Also please let me know if the issue was solved so that I can close this ticket.

[chuckie82]

It seems to work on my Mac OS X.

Does NOT work on Python 2.7, Linux (RHEL7 64 bit). (my_ana-1.0.7) (mypackages)[yoon82@psexport02 cbf]$ conda list cbf 1.1.1 np19py27_0 paulscherrerinstitute

In [1]: import cbf In [2]: a=cbf.read('a.cbf') In [3]: b=cbf.read('b.cbf') Segmentation fault (core dumped)

[simongregorebner]

would it be possible that you give me a dump of the installed packages on the my_ana machine and attach the generated file to this issue. While having this dump I would be able to re-generate your environment for testing.

conda list > packages.txt

[chuckie82]

(my_ana-1.0.7) (mypackages)[yoon82@psexport02 cbf]$ conda list

packages in environment at /reg/neh/home/yoon82/.conda/envs/my_ana-1.0.7:

# _nb_ext_conf 0.3.0 py27_0 defaults alabaster 0.7.9 py27_0 defaults anaconda-client 1.6.0 py27_0 defaults appdirs 1.4.0 py27_0 defaults args 0.1.0 py27_0 conda-forge astroid 1.4.7 py27_0 defaults babel 2.3.4 py27_0 defaults backports 1.0 py27_0 defaults backports_abc 0.5 py27_0 defaults bokeh 0.12.3 py27_0 defaults boost 1.57.0 4 defaults boto 2.43.0 py27_0 defaults bottleneck 1.0.0 np19py27_0 defaults bz2file 0.98 py27_0 defaults cairo 1.12.18 6 defaults cbf 1.1.1 np19py27_0 paulscherrerinstitute cffi 1.9.1 py27_0 defaults chest 0.2.3 py27_0 defaults click 6.6 py27_0 defaults clint 0.5.1 py27_0 conda-forge cloudpickle 0.2.1 py27_0 defaults clyent 1.2.2 py27_0 defaults configargparse 0.11.0 py27_0 defaults configparser 3.5.0 py27_0 defaults coverage 4.2 py27_0 defaults cryptography 1.6 py27_0 defaults cycler 0.10.0 py27_0 defaults cymem 1.31.2 py27_0 defaults cyordereddict 1.0.0 py27_1 conda-forge cython 0.25.2 py27_0 defaults dask 0.11.1 py27_0 defaults decorator 4.0.10 py27_1 defaults docutils 0.12 py27_2 defaults entrypoints 0.2.2 py27_0 defaults enum34 1.1.6 py27_0 defaults flask 0.11.1 py27_0 defaults fontconfig 2.11.1 6 defaults freeglut 2.8.1 0 defaults freetype 2.5.5 1 defaults funcsigs 1.0.2 py27_0 defaults functools32 3.2.3.2 py27_0 defaults future 0.16.0 py27_0 defaults futures 3.0.5 py27_0 defaults gensim 0.12.2 np19py27_0 defaults get_terminal_size 1.0.0 py27_0 defaults gevent 1.1.2 py27_0 defaults glib 2.43.0 1 defaults greenlet 0.4.10 py27_0 defaults guppy 0.1.10 py27_0 kalefranz h5netcdf 0.3.1 py27_0 conda-forge h5py 2.5.0 py27_mpi_0 file:///reg/g/psdm/sw/conda/channels/system-rhel7 harfbuzz 0.9.39 1 defaults hdf5 1.8.17 openmpi_101 file:///reg/g/psdm/sw/conda/channels/system-rhel7 heapdict 1.0.0 py27_1 defaults icu 54.1 0 defaults idna 2.1 py27_0 defaults imagesize 0.7.1 py27_0 defaults ipaddress 1.0.17 py27_0 defaults ipykernel 4.5.2 py27_0 defaults ipython 5.1.0 py27_0 defaults ipython_genutils 0.1.0 py27_0 defaults ipywidgets 5.2.2 py27_0 defaults itsdangerous 0.24 py27_0 defaults jasper 1.900.1 3 conda-forge jbig 2.1 0 defaults jedi 0.9.0 py27_1 defaults jinja2 2.8 py27_1 defaults joblib 0.9.4 py27_0 defaults jpeg 9b 0 conda-forge jsonschema 2.5.1 py27_0 defaults jupyter 1.0.0 py27_3 defaults jupyter_client 4.4.0 py27_0 defaults jupyter_console 5.0.0 py27_0 defaults jupyter_core 4.2.1 py27_0 defaults keras 1.1.0 py27_1 file:///reg/g/psdm/sw/conda/channels/external-rhel7 lazy-object-proxy 1.2.1 py27_0 defaults libffi 3.2.1 0 defaults libgfortran 3.0.0 1 defaults libiconv 1.14 0 defaults libpng 1.6.17 0 defaults libsodium 1.0.10 0 defaults libtiff 4.0.6 7 conda-forge libxml2 2.9.4 0 defaults llvm 3.3 0 defaults llvmlite 0.14.0 py27_0 defaults llvmpy 0.12.7 py27_0 defaults lmfit 0.8.3 py27_0 conda-forge locket 0.2.0 py27_1 defaults logbookclient 0.0.2 py27_0 file:///reg/g/psdm/sw/conda/channels/external-rhel7 markupsafe 0.23 py27_2 defaults matplotlib 1.4.3 np19py27_2 defaults meld3 1.0.2 py27_0 defaults mistune 0.7.3 py27_0 defaults mkl 11.3.3 0 defaults mkl-service 1.1.2 py27_2 defaults mock 2.0.0 py27_0 defaults mpi4py 2.0.0 py27_openmpi_101 file:///reg/g/psdm/sw/conda/channels/system-rhel7 mpmath 0.19 py27_1 defaults murmurhash 0.26.4 py27_0 defaults mysql 5.5.24 0 defaults mysql-connector-python 2.0.4 py27_0 defaults mysql-python 1.2.5 py27_0 defaults nb_anacondacloud 1.2.0 py27_0 defaults nb_conda 2.0.0 py27_0 defaults nb_conda_kernels 2.0.0 py27_0 defaults nbconvert 4.2.0 py27_0 defaults nbformat 4.2.0 py27_0 defaults nbpresent 3.0.2 py27_0 defaults ncurses 5.9 10 conda-forge ndarray 1.1.5 0 file:///reg/g/psdm/sw/conda/channels/psana-rhel7 neon 1.5.4 py27_0 defaults networkx 1.11 py27_0 defaults nltk 3.2.1 py27_0 defaults nose 1.3.7 py27_1 defaults notebook 4.2.3 py27_0 defaults numba 0.15.1 np19py27_0 defaults numexpr 2.4.4 np19py27_0 defaults numpy 1.9.3 py27_3 defaults numpydoc 0.6.0 py27_0 defaults opencv 2.4.12 np19py27_0 conda-forge openmpi 1.10.3 lsf_verbs_101 file:///reg/g/psdm/sw/conda/channels/system-rhel7 openssl 1.0.2j 0 defaults pandas 0.17.1 np19py27_0 defaults pango 1.39.0 1 defaults paramiko 2.0.2 py27_0 defaults partd 0.3.6 py27_0 defaults path.py 8.2.1 py27_0 defaults pathlib2 2.1.0 py27_0 defaults pbr 1.10.0 py27_0 defaults pcre 8.39 0 defaults pep8 1.7.0 py27_0 defaults pexpect 4.0.1 py27_0 defaults pickleshare 0.7.4 py27_0 defaults pillow 3.2.0 py27_1 conda-forge pip 9.0.1 py27_1 defaults pixman 0.32.6 0 defaults pkginfo 1.4.1 py27_0 defaults plotly 1.12.9 py27_0 defaults preshed 0.46.4 py27_0 defaults prompt_toolkit 1.0.9 py27_0 defaults protobuf 3.0.0b2 py27_0 conda-forge psana-conda 1.0.2 py27_1 file:///reg/g/psdm/sw/conda/channels/psana-rhel7 psutil 5.0.0 py27_0 defaults ptyprocess 0.5.1 py27_0 defaults py 1.4.31 py27_0 defaults py2cairo 1.10.0 py27_2 defaults pyasn1 0.1.9 py27_0 defaults pycairo 1.10.0 py27_0 defaults pycparser 2.17 py27_0 defaults pyflakes 1.3.0 py27_0 defaults pygments 2.1.3 py27_0 defaults pylint 1.5.4 py27_1 defaults pymc 2.3.4 np19py27_0 defaults pyopengl 3.1.1a1 np19py27_0 defaults pyparsing 2.0.3 py27_0 defaults pyqt 4.11.4 py27_4 defaults pyqtgraph 0.9.10 py27_1 defaults pytest 3.0.5 py27_0 defaults python 2.7.12 1 defaults python-dateutil 2.5.3 py27_0 defaults pytz 2016.10 py27_0 defaults pyyaml 3.12 py27_0 defaults pyzmq 16.0.2 py27_0 defaults qt 4.8.7 4 defaults qtawesome 0.3.3 py27_0 defaults qtconsole 4.2.1 py27_0 defaults qtpy 1.1.2 py27_0 defaults readline 6.2 0 conda-forge requests 2.11.1 py27_0 defaults requests-toolbelt 0.7.0 py27_0 conda-forge rope 0.9.4 py27_1 defaults scikit-beam 0.0.8 py27_0 scikit-beam scikit-image 0.11.3 np19py27_0 defaults scikit-learn 0.16.1 np19py27_0 defaults scipy 0.17.1 np19py27_1 defaults scons 2.3.0 py27_0 defaults seaborn 0.7.1 py27_0 defaults setuptools 27.2.0 py27_0 defaults simplegeneric 0.8.1 py27_1 defaults simplejson 3.10.0 py27_0 defaults singledispatch 3.4.0.3 py27_0 defaults sip 4.18 py27_0 defaults six 1.10.0 py27_0 defaults smart_open 1.3.5 py27_0 defaults snowballstemmer 1.2.1 py27_0 defaults sphinx 1.5 py27_0 defaults spyder 3.0.2 py27_0 defaults sqlalchemy 1.1.4 py27_0 defaults sqlite 3.13.0 0 defaults ssl_match_hostname 3.4.0.2 py27_1 defaults supervisor 3.3.1 py27_0 defaults swig 3.0.10 0 defaults sympy 1.0 py27_0 defaults system 5.8 2 defaults szip 2.1 101 file:///reg/g/psdm/sw/conda/channels/external-rhel7 tables 3.3.0 py27_100 file:///reg/g/psdm/sw/conda/channels/system-rhel7 tensorflow 0.10.0 py27_1 file:///reg/g/psdm/sw/conda/channels/external-rhel7 terminado 0.6 py27_0 defaults theano 0.8.2 py27_0 conda-forge thinc 5.0.8 py27_0 defaults tk 8.5.18 0 defaults toolz 0.8.1 py27_0 defaults tornado 4.4.2 py27_0 defaults tqdm 4.10.0 py27_0 conda-forge traitlets 4.3.1 py27_0 defaults twine 1.8.1 py27_1 conda-forge tzlocal 1.2.2 py27_0 conda-forge virtualenv 15.0.3 py27_0 defaults wcwidth 0.1.7 py27_0 defaults werkzeug 0.11.11 py27_1 defaults wheel 0.29.0 py27_0 defaults widgetsnbextension 1.2.6 py27_0 defaults workerpool 0.9.4 py27_1 defaults wrapt 1.10.8 py27_0 defaults xarray 0.8.2 py27_0 defaults xraylib 3.1.0 nppy27_0 scikit-beam xz 5.2.2 0 defaults yaml 0.1.6 0 defaults zeromq 4.1.5 0 defaults zlib 1.2.8 3 defaults

[simongregorebner]

It seems that there is a problem with your environment somehow. We tested the package on SLAC psexport with a clean anaconda environment. With this it worked without issues to open your files.

This is what we did:

# login
export PATH=/reg/g/xpp/xppcode/python/anaconda2/bin:$PATH
conda create -n mycbf numpy
source activate mycbf
conda install -c paulscherrerinstitute cbf

then opened an ipython shell

import cbf
>>> cbf.read('a.cbf')
Data(data=array([[2, 1, 1, ..., 0, 2, 0],
       [1, 2, 1, ..., 1, 1, 0],
       [0, 1, 0, ..., 0, 0, 1],
       ..., 
       [1, 0, 1, ..., 0, 1, 1],
       [1, 0, 2, ..., 1, 1, 0],
       [1, 0, 2, ..., 0, 1, 0]], dtype=uint32), metadata={'compression': u'x-CBF_BYTE_OFFSET', u'size_fastest_dimension': 2463, u'element_type': u'"signed 32-bit integer"', u'number_of_elements': 6224001, u'size_second_dimension': 2527, u'size_padding': 4095, u'element_byte_order': u'LITTLE_ENDIAN', u'size': 6224901, u'id': 1, 'md5': u'+fvsXgATqKmoQPs9TFYqNQ=='}, miniheader={})

(it worked with both of your files)

The environment packages were like this:

(mycbf) xxx@psexport01:~$ conda list 
# packages in environment at /reg/neh/home/xxx/.conda/envs/mycbf:
#
backports                 1.0                      py27_0  
cbf                       1.1.1                np19py27_0    paulscherrerinstitute
decorator                 4.0.10                   py27_1  
enum34                    1.1.6                    py27_0  
get_terminal_size         1.0.0                    py27_0  
ipython                   5.1.0                    py27_0  
ipython_genutils          0.1.0                    py27_0  
mkl                       11.3.3                        0  
numpy                     1.9.3                    py27_3  
openssl                   1.0.2j                        0  
path.py                   8.2.1                    py27_0  
pathlib2                  2.1.0                    py27_0  
pexpect                   4.0.1                    py27_0  
pickleshare               0.7.4                    py27_0  
pip                       9.0.1                    py27_1  
prompt_toolkit            1.0.9                    py27_0  
ptyprocess                0.5.1                    py27_0  
pygments                  2.1.3                    py27_0  
python                    2.7.12                        1  
readline                  6.2                           2  
setuptools                27.2.0                   py27_0  
simplegeneric             0.8.1                    py27_1  
six                       1.10.0                   py27_0  
sqlite                    3.13.0                        0  
tk                        8.5.18                        0  
traitlets                 4.3.1                    py27_0  
wcwidth                   0.1.7                    py27_0  
wheel                     0.29.0                   py27_0  
zlib                      1.2.8                         3

Please check again your environment variables and settings.

[chuckie82]

We recently moved our software stack to conda, and something must have been mixed up. This works now on my Linux RHEL7 machine.

Thanks again for looking into this.

[simongregorebner]

No worries. Good to know that things work now.