Returned array's size error

[FabricioS]

Hi, trying to apply the GstSatbilizer plugin to a bunch of jpg images, I got the following error: ERREUR : de l'élément /GstPipeline:pipeline0/flow_finder+OpticalFlowFinder:flow_finder+opticalflowfinder0 : total size of new array must be unchanged Information de débogage supplémentaire : /home/fab/Boulot/test/PP_stab/GstStabilizer/python/flow_finder.py(96): _chain (): /GstPipeline:pipeline0/flow_finder+OpticalFlowFinder:flow_finder+opticalflowfinder0: <type 'exceptions.ValueError'>: total size of new array must be unchanged

Any hint?

Complete verbose session $ gst-launch -v multifilesrc location="im/%d.JPG" index=1 num-buffers=40 caps="image/jpeg,framerate=(fraction)6/1" ! jpegdec ! tee name=tee tee. ! ffmpegcolorspace ! opticalflowfinder ! opticalflowrevert name=mux tee. ! ffmpegcolorspace ! mux. mux. ! ffmpegcolorspace ! jpegenc ! multifilesink location="im_out/%d.JPG"

Définition du pipeline à PAUSED... /GstPipeline:pipeline0/GstMultiFileSrc:multifilesrc0.GstPad:src: caps = image/jpeg, depth=(int)24, bpp=(int)24, endianness=(int)4321, red_mask=(int)16711680, green_mask=(int)65280, blue_mask=(int)255, width=(int)1, height=(int)1, framerate=(fraction)6/1, sof-marker=(int)0 Le pipeline est en phase de PREROLL... /GstPipeline:pipeline0/GstMultiFileSrc:multifilesrc0.GstPad:src: caps = image/jpeg, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstJpegDec:jpegdec0.GstPad:sink: caps = image/jpeg, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstJpegDec:jpegdec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstTee:tee: alloc-pad = (GstPad) src1 /GstPipeline:pipeline0/GstTee:tee.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstTee:tee.GstPad:src1: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstTee:tee.GstPad:src0: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstTee:tee.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp1.GstPad:src: caps = video/x-raw-rgb, depth=(int)24, width=(int)2250, height=(int)1500, framerate=(fraction)6/1, bpp=(int)24, endianness=(int)4321, red_mask=(int)16711680, green_mask=(int)65280, blue_mask=(int)255 /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp1.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/flow_revert+OpticalFlowRevert:mux.GstPad:mainsink: caps = video/x-raw-rgb, depth=(int)24, width=(int)2250, height=(int)1500, framerate=(fraction)6/1, bpp=(int)24, endianness=(int)4321, red_mask=(int)16711680, green_mask=(int)65280, blue_mask=(int)255 /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-gray, depth=(int)8, width=(int)2250, height=(int)1500, framerate=(fraction)6/1, bpp=(int)8 /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)2250, height=(int)1500, framerate=(fraction)6/1 /GstPipeline:pipeline0/flow_finder+OpticalFlowFinder:flow_finder+opticalflowfinder0.GstPad:sink: caps = video/x-raw-gray, depth=(int)8, width=(int)2250, height=(int)1500, framerate=(fraction)6/1, bpp=(int)8 ERREUR : de l'élément /GstPipeline:pipeline0/flow_finder+OpticalFlowFinder:flow_finder+opticalflowfinder0 : total size of new array must be unchanged Information de débogage supplémentaire : /home/fab/Boulot/test/PP_stab/GstStabilizer/python/flow_finder.py(96): _chain (): /GstPipeline:pipeline0/flow_finder+OpticalFlowFinder:flow_finder+opticalflowfinder0: <type 'exceptions.ValueError'>: total size of new array must be unchanged ERREUR : le pipeline refuse de passer en phase de préparation. Définition du pipeline à NULL... /GstPipeline:pipeline0/flow_revert+OpticalFlowRevert:mux.GstPad:mainsink: caps = NULL /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp1.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp1.GstPad:sink: caps = NULL /GstPipeline:pipeline0/flow_finder+OpticalFlowFinder:flow_finder+opticalflowfinder0.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstTee:tee.GstPad:src1: caps = NULL /GstPipeline:pipeline0/GstTee:tee.GstPad:src0: caps = NULL /GstPipeline:pipeline0/GstTee:tee.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstJpegDec:jpegdec0.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstJpegDec:jpegdec0.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstMultiFileSrc:multifilesrc0.GstPad:src: caps = NULL Libération du pipeline...

[FabricioS]

Some more infos captured in OpticalFlowFinder._chain before calling img_of_buf : buffer caps : video/x-raw-gray, depth=(int)8, width=(int)2250, height=(int)1500, framerate=(fraction)6/1, bpp=(int)8 buffer.size : 3378000

but img_of_buf is trying to reshape it into (height, width, channels)=(1500, 2250, 1), i.e. a 3375000 sized array, the same as the original images.

Any idea about what are the 3000 extra elements?

[FabricioS]

Inspecting the buffer entering the _chain method and saving it, it contains two extras columns on the right side of the image.

Here the added lines, for completeness

def _chain(self, pad, buf):
    import numpy
    from scipy import misc
    tmp = numpy.frombuffer(buf, dtype=numpy.uint8)
    tmp = tmp.reshape((buf.caps[0]['height'], -1))
    misc.imsave('test.png', tmp)
   ...

[FabricioS]

Discussing on IRC#gstreamer, it appears that the width dimensions is padded due to alignment rules (it has to be a multiple of 4 on my machine for this format). The following functions may be of some interest to take this into account:

• gst_video_format_get_row_stride()
• gst_video_info_from_caps()
• GST_VIDEO_INFO_PLANE_STRIDE() and GST_VIDEO_INFO_COMP_STRIDE()

Unfortunately, the plugin is based on a python wrapped gstreamer, not the "Gobject Introspect"-ed one, and the former does not wrap these function (at least they are not listed in "import gst; dir(gst)")