(-215:Assertion failed) sz.width % 2 == 0 && sz.height % 3 == 0 in function 'CvtHelper'

[Rapha149]

This is the output of ./deepseg -d -d -c /dev/video0 -v /dev/video1:

deepseg v0.2.0
(c) 2021 by floe@butterbrot.org
https://github.com/floe/deepbacksub
debug:  2
ccam:   /dev/video0
vcam:   /dev/video1
width:  640
height: 480
flip_h: no
flip_v: no
threads:2
back:   (none)
model:  models/segm_full_v679.tflite

vid_format->type                = 2
vid_format->fmt.pix.width       = 640
vid_format->fmt.pix.height      = 480
vid_format->fmt.pix.pixelformat = 1448695129
vid_format->fmt.pix.sizeimage   = 614400
vid_format->fmt.pix.field       = 1
vid_format->fmt.pix.bytesperline= 1280
vid_format->fmt.pix.colorspace  = 8

tensor #0: 1
tensor #0: 144
tensor #0: 256
tensor #0: 3
tensor #244: 1
tensor #244: 144
tensor #244: 256
tensor #244: 2
Invalid MIT-MAGIC-COOKIE-1 keyQSettings::value: Empty key passed
QSettings::value: Empty key passed
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(3.4.9) /home/abuild/rpmbuild/BUILD/opencv-3.4.9/modules/imgproc/src/color.simd_helpers.hpp:104: error: (-215:Assertion failed) sz.width % 2 == 0 && sz.height % 3 == 0 in function 'CvtHelper'

Thanks in advance!

[phlash]

Not sure if the Invalid MIT-MAGIC-COOKIE-1 key is the problem or the assertion.. if you are trying to run this remotely, you may want to remove the -d -d to avoid the application trying to display an image (but not finding a valid .Xauthority file). With the display issue out of the way we have a chance :)

If there is still a problem, and you are happy to try a debugger.. then running the application in gdb and collecting a backtrace would be useful:

% gdb ./deepseg
gdb> run -c /dev/video0 -v /dev/video1
[wait for crash]
gdb> bt

[Rapha149]

I don't know what Invalid MIT-MAGIC-COOKIE-1 key exactly is, but I have a temporarily fix for it. If I do the fix, the output is the same, just without the Invalid MIT-MAGIC-COOKIE-1 key. That's the output of the gdb commands when I do bt:

#0  0x00007ffff2ad9520 in raise () at /lib64/libc.so.6
#1  0x00007ffff2adab01 in abort () at /lib64/libc.so.6
#2  0x00007ffff366d016 in  () at /usr/lib64/libstdc++.so.6
#3  0x00007ffff367888c in  () at /usr/lib64/libstdc++.so.6
#4  0x00007ffff36788f7 in  () at /usr/lib64/libstdc++.so.6
#5  0x00007ffff3678ba9 in  () at /usr/lib64/libstdc++.so.6
#6  0x00007ffff3bd4072 in cv::error(cv::Exception const&) () at /usr/lib64/libopencv_core.so.3.4
#7  0x00007ffff3bd548c in cv::error(int, cv::String const&, char const*, char const*, int) () at /usr/lib64/libopencv_core.so.3.4
#8  0x00007ffff41b4a4a in  () at /usr/lib64/libopencv_imgproc.so.3.4
#9  0x00007ffff4234a03 in  () at /usr/lib64/libopencv_imgproc.so.3.4
#10 0x00007ffff424164e in  () at /usr/lib64/libopencv_imgproc.so.3.4
#11 0x00007ffff41f2c7d in cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int) () at /usr/lib64/libopencv_imgproc.so.3.4
#12 0x0000000000408da5 in main(int, char**) (argc=<optimized out>, argv=<optimized out>) at deepseg.cc:326

[phlash]

OK, thanks, that confirms the crash is still in the conversion step. I've been digging for camera formats and fourcc codes (the 'YU12' in your logs), which seems to indicate that I guessed wrong earlier with NV12. Please try with: cv::cvtColor(in_u8_yuv,in_u8_rgb,CV_YUV2RGB_I420); this is according to: http://wiki.oz9aec.net/index.php/Pixel_formats I will continue to look into OpenCV source code and attempt to discover how it automatically converts to RGB during capture when asked (also how my PR fixes this for me, but YMMV), as this will form the basis of a robust fix.

[Rapha149]

Same error :neutral_face:

[phlash]

Well, that's weird. As far as I can see, OpenCV's automatic conversion uses the same call, assuming your camera really is providing YU12 (aka V4L2_PIX_FMT_YUV420) format frames: https://github.com/opencv/opencv/blob/64e6cf9fe50ef25d317d3529a42d4862b93f002a/modules/videoio/src/cap_v4l.cpp#L1546 (note that COLOR_YUV2BGR_I420 is an alias for COLOR_YUV2BGR_IYUV)

We may need to try a different approach, letting OpenCV convert to RGB during capture, I'll do that in a separate fix branch and let you know when it's ready to test..

[Rapha149]

Really weird... Thank you for your efforts, and sorry for the circumstances!

[phlash]

No worries, I've just pushed a branch which uses OpenCV to convert input to BGR, rather than try and do it ourselves. Please give it a go and let me know here.. branch issue47

[Rapha149]

It worked now :slightly_smiling_face: But: is this normal?

[Rapha149]

Ok I looked at #29 and #30. I used now another model and it does not log anymore, but I'm still just green, should I open a new issue?

[phlash]

Yay!, OK, I'll get the fix pushed into main. That is normal for low light conditions and no background image selected.

[Rapha149]

As I said, I used another model and I also tried out to select a background image. I don't lag anymore, but I'm still just green:

[phlash]

OK, I have reproduced this locally now, I too am a green blob with this patch.. I have missed something...

[Rapha149]

I don't want to be impatient, but how is it going?

[BenBE]

It's being handled in #48 and looks like it's good to go. There were some performance considerations that had to be discussed and been decided upon first.

That aside, none of the people working on this project do so as their day-job and in fact are usually quite busy with other tasks that life demands of them. Please be patient until we are sure that this patch doesn't break things for other people and thus decide to go ahead and finally integrate it.

[Rapha149]

Thanks for the answer and yes I know and appreciate that you're doing this here voluntarily. I just wanted to get a little status update and didn't want to sound impatient. I apologize if my message sounded like that :neutral_face: