imxvideoconvert_ipu convertion got problem with imxv4l2src?

[trantu87]

Hi there,

I have tested the the pipeline for cropping and rotation of imxvideoconvert_ipu on kernel 4.1.15 imx6qsabresd board. I got some issues that would like to confirm as below:

1. Max size supported for rotation is only 1024X1024.

Can anybody confirm me whether my understand is correct?

1. There is a problem when I try to cropping the data got from imxv4l2src. Below is my pipeline:

a) gst-launch-1.0 -v imxv4l2src device=/dev/video1 ! video/x-raw, format=I420,width=1920, height=1080, framerate=30/1 ! imxvideoconvert_ipu ! video/x-raw, width=1024, height=768 ! vpuenc_h264 gop-size=30 bitrate=2048 ! h264parse ! rtph264pay name=pay0 pt=96 timestamp-offset=0 ! udpsink host=192.168.0.100 port=1234 &

Please see my attached file for clearing about the issue with the image from client side.

IMG_20160810_144709.zip

If I try the same pipeline but change from imxv4l2src to videotestsrc, the displayed image from client side is very smooth and beautiful.

b) gst-launch-1.0 -v videotestsrc ! video/x-raw, format=I420,width=1920, height=1080, framerate=30/1 ! imxvideoconvert_ipu ! video/x-raw, width=1024, height=768 ! vpuenc_h264 gop-size=30 bitrate=2048 ! h264parse ! rtph264pay name=pay0 pt=96 timestamp-offset=0 ! udpsink host=192.168.0.100 port=1234 &

Please share me any hint where the issue comes from case a)

I would like to thank a lot for any support on this issue.

[dv1]

Hi, sorry about the inactivity, other issues are currently blocking me.

1. 1024x1024 is indeed the limit, but only in certain cases. If for example no rotation and no deinterlacing is done, then the driver apparently tiles larger frames internally on its own. But this is very inconsistent - and undocumented.
2. imx4vl2src? Either you are combining an element from NXP's plugins with gstreamer-imx, or you are using an older version of gstreamer-imx, since in newer verisons of gstreamer-imx, the v4l element has been renamed to "imxv4l2videosrc" to avoid collisions with imx4vl2src from the NXP plugins. What is the version number of you gstreamer-imx installation?

[trantu87]

Hi dv1,

Actually I am using the plugin in fsl-plugin developed by NXP not the gstreamer-imx plugin. I already asked the question in NXP community but they advised me to follow here for other information. I also tried to upgrade and enable gstreamer-imx plugins (version 0.12.2) in my environment for checking if the issue is come from NXP plugin only. As the result I saw that imxv4l2videosrc can work with imxvideoconvert_ipu in resizing, do not have the green band in the bottom of image.

gst-launch-1.0 -v imxv4l2videosrc device=/dev/video1 imx-capture-mode=5 ! imxvideoconvert_ipu ! video/x-raw, width=1024, height=768 ! vpuenc_h264 gop-size=30 bitrate=4096 ! h264parse ! rtph264pay name=pay0 pt=96 timestamp-offset=0 ! udpsink host=192.168.2.12 port=1234

I would like to close the issue here to further investigate why the issue happens with NXP plugins in updated version.

Thanks for your reply dv1 :-)

[dv1]

Please re-run the imxv4l2videosrc pipeline with the environment variable GST_DEBUG set to "2,imx:9" . Also, generate a dot graph so we can see the structure of the pipeline: set the GST_DEBUG_DUMP_DOT_DIR environment variable to "/tmp/" , and run the gst-launch-1.0 line. This produces .dot files in /tmp/ . copy the file that ends with "gst-launch.PAUSED_PLAYING.dot" to your computer, and run:

dot -Tpng -o graph.png <the dot file you copied>

(You might need to install the graphviz package first.)

Then upload the resulting PNG, for example to imgur, and post the link.

[dv1]

Update: I have observed the green band with I420 input data. This appears to be an IPU problem. I modified imxv4l2videosrc to also offer UYVY as output format. With this, the green band is gone. Also, please notice that the NXP plugins won't work efficiently with the gstreamer-imx plugins because they don't recognize the other's physical memory information (GstMemory subclasses, GstMeta subclasses containing physical addresses), so frames will be copied by the CPU, not by DMA.

[trantu87]

Hi dv1,

Thank you for your information. I have checked with NXP plugins (capture, resize, encode). The green band will happens with I420 and NV12 format. It is fine if I used the UYVY format as your suggestion. There is something wrong in NXP plugins in the updated version because older version can be worked fine with IPU.

From your updated code I understand that currently we change the format from I420 to UYVY to avoid problem with IPU, it is not the root cause of issue for I420. Is my understanding correct, dv1?

[dv1]

The root cause seems to be a problem with the plane offsets. However, I cannot see what exactly is wrong with them (or if there is anything wrong at all). My impression is that perhaps I am forgetting something when using the deinterlacer. The problem is though that while the VDIC is properly documented in the iMX6 reference manual, the IPU driver - which controls the VDIC - is undocumented, and it is very difficult to understand its source code, so it is unclear how it actually controls the VDIC.

[trantu87]

Hi dv1,

I have one question related to bit depth. I420 with 12 bits mean format 4:2:0, and the define of UYVY is 16 bits format is 4:2:2. Is this affect to the result?

[dv1]

It does, since the vertical chroma resolution is 2x higher. 4:2:2 is potentially nicer for deinterlacing, because the chroma pixels for the odd and even scanlines aren't shared.

[trantu87]

Hi dv1,

But the default setting passing to IPU is no deinterlace. It is processed as progressive picture. So how can it influenced? :-(