seg-faults when profiled ...

[kif]

In ipython (2 or 3): import cbf %timeit cbf.read(a_cbf_file)

python3: malloc.c:2372: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) * 2])) - builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 (sizeof(size_t))) - 1)) & ~((2 *(sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long) old_end & pagemask) == 0)' failed. Aborted

[simongregorebner]

• What happens if you profile the cbf file in the examples folder that comes with this repository?

import cbf
%timeit cbf.read('examples/example.cbf')

• What happens if you are just open one of your cbf files without profiling (i.e. just a _cbf.read(a_cbffile) )?

[kif]

On Thu, 19 May 2016 05:05:25 -0700 Simon Gregor Ebner notifications@github.com wrote:

• What happens if you profile the cbf file in the examples folder that comes with this repository?

I didn't ... I will try one I get back on this computer ... The image I tested was part of the demo dataset dectris was shipping out to advertise Pilatus. the file is run2_1_00148.cbf

import cbf
%timeit cbf.read('examples/example.cbf')

• What happens if you are just open one of your cbf files without profiling (i.e. just a _cbf.read(a_cbffile) )?

This works and the results were looking correct.

Currently I am looking for a solution to process pilatus 2M images at full speed which enforces the read time <4ms in practice 1ms being the target.

Cheers,

Jérôme

[kif]

The file you provide as test works but not: http://www.silx.org/pub/fabio/testimages/run2_1_00148.cbf

In [3]: r=cbf.read("example(1).cbf")

In [4]: r Out[4]: Data(data=array([[0, 2, 1, ..., 1, 2, 1], [0, 0, 1, ..., 0, 2, 1], [1, 2, 3, ..., 2, 0, 0], ..., [1, 2, 0, ..., 2, 1, 0], [1, 1, 2, ..., 0, 0, 2], [1, 1, 1, ..., 1, 2, 1]], dtype=uint32), metadata={u'size_padding': 4095, u'element_type': u'"signed 32-bit integer"', u'number_of_elements': 2476525, u'size_second_dimension': 1679, u'element_byte_order': u'LITTLE_ENDIAN', u'size': 2478005, u'id': 1, u'size_fastest_dimension': 1475})

In [5]: %timeit r=cbf.read("example(1).cbf") 100 loops, best of 3: 5.85 ms per loop

In [6]: import fabio

In [8]: %timeit fabio.cbfimage.CbfImage().read("example(1).cbf") 100 loops, best of 3: 9.44 ms per loop

In [10]: %timeit fabio.cbfimage.CbfImage().read("example(1).cbf", check_MD5=False) 100 loops, best of 3: 5.73 ms per loop

In [11]: fname="/home/kieffer/workspace/fabio/testimages/run2_1_00148.cbf"

In [13]: %timeit fabio.cbfimage.CbfImage().read(fname, check_MD5=False) 10 loops, best of 3: 22.7 ms per loop

In [14]: r=cbf.read(fname) python: malloc.c:2372: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) * 2])) - builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 (sizeof(size_t))) - 1)) & ~((2 *(sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long) old_end & pagemask) == 0)' failed. Aborted

[simongregorebner]

I (probably) tracked down the problem down to a problem in detecting the start of the binary part of the file. I will go into this in detail this afternoon ...

[simongregorebner]

It was actually an error in interpreting the length of the binary part that actually contains data