Getting segmentation fault for 3D crf

[pianoza]

Hi, I'm trying to run pydensecrf for 3D case, but getting segmentation fault when I call inference. Here's how I'm running it (not sure if this is the correct way):

# probs of shape 3d image per class: Nb_classes x Height x Width x Depth
# assume the image has shape (69, 51, 72)
shape = image.shape
new_image = np.empty(shape)
d = dcrf.DenseCRF(np.prod(shape), probs.shape[0])
U = unary_from_softmax(probs)
d.setUnaryEnergy(U)
feats = create_pairwise_gaussian(sdims=(1.0, 1.0, 1.0), shape=shape)
d.addPairwiseEnergy(feats, compat=3, kernel=dcrf.FULL_KERNEL, normalization=dcrf.NORMALIZE_SYMMETRIC)
Q = d.inference(5) 
new_image = np.argmax(Q, axis=0).reshape((shape[0], shape[1],shape[2]))

..and here is the backtrace:

#0  __memcpy_avx_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:116
#1  0x00007ffec011e4f9 in memcpy (__len=60, __src=0x15e11fdc, __dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string3.h:53
#2  Permutohedral::sseCompute (this=0x432e330, out=0x14ded4d0, in=0x14ded4d0, value_size=15, reverse=false) at pydensecrf/densecrf/src/permutohedral.cpp:546
#3  0x00007ffec0117f17 in DenseKernel::filter (transpose=false, in=..., out=..., this=0x432e320) at pydensecrf/densecrf/src/pairwise.cpp:74
#4  DenseKernel::apply (Q=..., out=..., this=0x432e320) at pydensecrf/densecrf/src/pairwise.cpp:124
#5  PairwisePotential::apply (this=0x95187c0, out=..., Q=...) at pydensecrf/densecrf/src/pairwise.cpp:174
#6  0x00007ffec010ee1f in DenseCRF::inference (this=0x4c10b50, n_iterations=n_iterations@entry=3) at pydensecrf/densecrf/src/densecrf.cpp:125
#7  0x00007ffec00fd23c in __pyx_pf_10pydensecrf_8densecrf_8DenseCRF_10inference (__pyx_v_self=0x7ffec067e378, __pyx_v_niter=3) at pydensecrf/densecrf.cpp:3364
#8  __pyx_pw_10pydensecrf_8densecrf_8DenseCRF_11inference (__pyx_v_self=0x7ffec067e378, __pyx_arg_niter=<optimized out>) at pydensecrf/densecrf.cpp:3340
#9  0x00000000004ca15e in PyEval_EvalFrameEx ()
#10 0x00000000004c2765 in PyEval_EvalCodeEx ()
#11 0x00000000004c2509 in PyEval_EvalCode ()
#12 0x00000000004f1def in ?? ()
#13 0x00000000004ec652 in PyRun_FileExFlags ()
#14 0x00000000004eae31 in PyRun_SimpleFileExFlags ()
#15 0x000000000049e14a in Py_Main ()
#16 0x00007ffff7811830 in __libc_start_main (main=0x49dab0 <main>, argc=2, argv=0x7fffffffe218, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffe208) at ../csu/libc-start.c:291
#17 0x000000000049d9d9 in _start ()

Could you please help me to figure out what am I doing wrong?

[lucasb-eyer]

Hi!

I'm currently working on a deadline and will be too busy to look into this for the next couple weeks. Let me give you a few pointers though:

A segfault usually comes from wrong use of the API: if you'd use the C++ code in the same way, it'd segfault too. Your image is non-square, so my guess would be some shape in a wrong order somewhere. Another quick guess: is the ndarray C-contiguous? Someone recently contributed an example of non-RGB-image data, which I turned into a tutorial-style notebook. You could cross-check with it to maybe find the mistake.

In any case, if you find the mistake before I get to it, please let me know so I may add safeguards into the PyDenseCRF API for future users of it not to fall into the same pitfall.

Of course, it might also be a bug with the wrapper, which may take some time to find.

[pianoza]

Hi, I found my mistake: the shape of the probs and the image.shape were not the same, I had one of the axes swapped in the probs. I think it would be cool to have an assert in the function, but anyways thanks, your comment was helpful!

[lucasb-eyer]

Thank you for posting an update! I will definitely add an assert as soon as I get some down time!

[lucasb-eyer]

FYI, I added a little more protection from this in the above commit, thanks again for your feedback!

[Thatfreesky]

Hi @pianoza , I recently also want to use pydensecrf to deal with 3D data. Can you share me your solution? From the commit of lucasb, as below, I still can not know how to use densecrf on 3D data. Thanks.

SHAPE, NLABELS = (69, 51, 72), 5
probs = np.random.randn(NLABELS, 68, 50).astype(np.float32)  # WRONG shape here
probs /= probs.sum(axis=0, keepdims=True)

d = dcrf.DenseCRF(np.prod(SHAPE), NLABELS)

d.setUnaryEnergy(unary_from_softmax(probs))  # THIS SHOULD THROW and not crash later
feats = create_pairwise_gaussian(sdims=(1.0, 1.0, 1.0), shape=SHAPE)
d.addPairwiseEnergy(feats, compat=3, kernel=dcrf.FULL_KERNEL, normalization=dcrf.NORMALIZE_SYMMETRIC)

Q = d.inference(5)
new_image = np.argmax(Q, axis=0).reshape(SHAPE)

[pianoza]

Hi @Thatfreesky, The way I used it for a 3D data was exactly the same as in my first comment.

[MiguelAngeloMartinsRibeiro]

Hi @Thatfreesky, The way I used it for a 3D data was exactly the same as in my first comment.

Hi @pianoza and @Thatfreesky I'm trying to use it as a 3D dense CRF as in this paper from the same authors https://www.sciencedirect.com/science/article/pii/S1361841516301839

I want to apply it to 3D RGB images to get a temporal correlation o videos and I'm having troubles cause the 3D version doesn't exist in a python library

I wanna know if that's what you're trying to do as well and if you can help me through it

Thank you