Open masip85 opened 2 years ago
dusty-nv
commented
2 years ago Hi @masip85, my guess is some issue or connectivity on the RTSP source side. It also appears this RTSP source has high resolution of 3072x1728. Maybe you want to try lower resolution at first?
If you just run a normal gstreamer pipeline (without the hardware-accelerated omx decoder), does it work?
masip85
commented
2 years ago Hello I am using another camera of our collection, and now is getting frames, but bad quality (blurry/huge pixels zone). I've tried default encoding and mpeg4 too.
Other issue is that if I set into gst-launch-1.0 the same parameters you use, video is recorded but there are no logs at each received frame:
Original video-viewer log:
(video-viewer:2624): GStreamer-CRITICAL **: 13:14:57.868: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
(video-viewer:2624): GStreamer-CRITICAL **: 13:14:57.868: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
(video-viewer:2624): GStreamer-CRITICAL **: 13:14:57.868: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
(video-viewer:2624): GStreamer-CRITICAL **: 13:14:57.868: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
(video-viewer:2624): GStreamer-CRITICAL **: 13:14:57.868: gst_structure_get_string: assertion 'structure != NULL' failed
(video-viewer:2624): GStreamer-CRITICAL **: 13:14:57.868: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Allocating new output: 3072x1728 (x 15), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 3072, nFrameHeight = 1728
[gstreamer] gstDecoder -- onPreroll()
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer changed state from NULL to READY ==> manager
[gstreamer] gstreamer changed state from READY to PAUSED ==> manager
[gstreamer] gstreamer changed state from NULL to READY ==> rtpssrcdemux1
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtpssrcdemux1
[gstreamer] gstreamer changed state from NULL to READY ==> rtpsession1
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtpsession1
[gstreamer] gstreamer changed state from NULL to READY ==> funnel2
[gstreamer] gstreamer changed state from READY to PAUSED ==> funnel2
[gstreamer] gstreamer changed state from NULL to READY ==> funnel3
[gstreamer] gstreamer changed state from READY to PAUSED ==> funnel3
[gstreamer] gstreamer changed state from NULL to READY ==> rtpstorage1
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtpstorage1
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer changed state from NULL to READY ==> udpsink2
[gstreamer] gstreamer changed state from READY to PAUSED ==> udpsink2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> udpsink2
[gstreamer] gstreamer changed state from NULL to READY ==> fakesrc1
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> fakesrc1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> fakesrc1
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtpssrcdemux1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtpstorage1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtpsession1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> funnel2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> funnel3
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> manager
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> udpsrc2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> udpsrc2
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> udpsrc3
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> udpsrc3
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer changed state from NULL to READY ==> rtpptdemux1
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtpptdemux1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtpptdemux1
[gstreamer] gstreamer changed state from NULL to READY ==> rtpjitterbuffer1
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtpjitterbuffer1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtpjitterbuffer1
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer message stream-start ==> pipeline0
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer mysink taglist, video-codec=(string)"H.264\ \(Main\ Profile\)";
[gstreamer] gstBufferManager -- map buffer size was less than max size (1008 vs 7962624)
[gstreamer] gstBufferManager recieve caps: video/x-raw(memory:NVMM), format=(string)NV12, width=(int)3072, height=(int)1728, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
[gstreamer] gstBufferManager -- recieved first frame, codec=h264 format=nv12 width=3072 height=1728 size=7962624
[gstreamer] gstBufferManager -- recieved 3072x1728 frame (7962624 bytes)
[gstreamer] gstBufferManager -- recieved NVMM memory
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer message async-done ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0
[gstreamer] gstBufferManager -- recieved 3072x1728 frame (7962624 bytes)
[gstreamer] gstBufferManager -- recieved 3072x1728 frame (7962624 bytes)
nvbuf_utils: dmabuf_fd 1071 mapped entry NOT found
nvbuf_utils: NvReleaseFd Failed... Exiting...
[gstreamer] gstBufferManager -- recieved 3072x1728 frame (7962624 bytes)
nvbuf_utils: dmabuf_fd 1073 mapped entry NOT found
....
video-viewer: captured 106 frames (3072 x 1728)
[gstreamer] gstEncoder -- appsrc requesting data (4096 bytes)
^Creceived SIGINT
[gstreamer] gstBufferManager -- recieved 3072x1728 frame (7962624 bytes)
video-viewer: captured 107 frames (3072 x 1728)
video-viewer: shutting down...
[gstreamer] gstDecoder -- stopping pipeline, transitioning to GST_STATE_NULL
[gstreamer] gstDecoder -- onPreroll()
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> capsfilter1
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> omxh264dec-omxh264dec0
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> h264parse1
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> rtph264depay1
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> queue0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> rtspsrc0
[gstreamer] gstreamer changed state from PLAYING to PAUSED ==> pipeline0
[gstreamer] gstEncoder -- appsrc requesting data (4096 bytes)
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstDecoder -- pipeline stopped
[gstreamer] gstEncoder -- shutting down pipeline, sending EOS
[gstreamer] gstEncoder -- appsrc requesting data (4096 bytes)
[gstreamer] gstEncoder -- transitioning pipeline to GST_STATE_NULL
[gstreamer] gstEncoder -- pipeline stopped
^Creceived SIGINTOk,that is correct. Now I get this parameters and insert them into gst-launch-1.0:
root@263a942a2f44:~/repos/jetson-utils/build/aarch64/bin# gst-launch-1.0 -e rtspsrc location=rtsp://X:X@192.168.121.214 latency=2000 ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12" ! omxh264enc bitrate=4000000 ! video/x-h264 ! h264parse ! qtmux ! filesink location=lolo.mp4
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://X:X@192.168.121.214
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
(gst-launch-1.0:2667): GStreamer-CRITICAL **: 13:22:09.925: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
(gst-launch-1.0:2667): GStreamer-CRITICAL **: 13:22:09.925: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
(gst-launch-1.0:2667): GStreamer-CRITICAL **: 13:22:09.925: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
(gst-launch-1.0:2667): GStreamer-CRITICAL **: 13:22:09.925: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
(gst-launch-1.0:2667): GStreamer-CRITICAL **: 13:22:09.926: gst_structure_get_string: assertion 'structure != NULL' failed
(gst-launch-1.0:2667): GStreamer-CRITICAL **: 13:22:09.926: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Allocating new output: 3072x1728 (x 15), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 3072, nFrameHeight = 1728
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 40
^Chandling interrupt.
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
Waiting for EOS...
Got EOS from element "pipeline0".
EOS received - stopping pipeline...
Execution ended after 0:00:21.485239917
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...As you can see,I don't see how frames are being received,but, video is recorded as is expected. But then I want to process frames in openCV as they are received... and.. I use:
Camera input: rtspsrc location=rtsp://X:X@192.168.121.214 latency=2000 ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12" ! appsink drop=true sync=false
Thread: rtspsrc location=rtsp://X:X@192.168.121.214 latency=2000 ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12" ! appsink drop=true sync=false started
data->gstreamer: 1
stream_mode: 1800
Listening to tcp://0.0.0.0:8887 waiting for COMPSs ack to start
(edge:2591): GStreamer-CRITICAL **: 13:07:41.567: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
(edge:2591): GLib-GObject-WARNING **: 13:07:41.573: invalid cast from 'GstAppSink' to 'GstBin'
(edge:2591): GStreamer-CRITICAL **: 13:07:41.573: gst_bin_iterate_elements: assertion 'GST_IS_BIN (bin)' failed
(edge:2591): GStreamer-CRITICAL **: 13:07:41.573: gst_iterator_next: assertion 'it != NULL' failed
(edge:2591): GStreamer-CRITICAL **: 13:07:41.573: gst_iterator_free: assertion 'it != NULL' failed
[ WARN:0@9.260] global /root/repos/opencv/modules/videoio/src/cap_gstreamer.cpp (1226) open OpenCV | GStreamer warning: cannot find appsink in manual pipeline
[ WARN:0@9.260] global /root/repos/opencv/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been createdAnd what I get is this, nothing else at all,can't use ctrl+c and I have to kill process from top.
Camera input: rtspsrc location=rtsp://X:X@192.168.121.214 latency=2000 ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12" ! appsink drop=true sync=false
Thread: rtspsrc location=rtsp://X:X@192.168.121.214 latency=2000 ! queue ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12" ! appsink drop=true sync=false started
data->gstreamer: 1
stream_mode: 1800
(edge:2591): GStreamer-CRITICAL **: 13:07:41.567: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
(edge:2591): GLib-GObject-WARNING **: 13:07:41.573: invalid cast from 'GstAppSink' to 'GstBin'
(edge:2591): GStreamer-CRITICAL **: 13:07:41.573: gst_bin_iterate_elements: assertion 'GST_IS_BIN (bin)' failed
(edge:2591): GStreamer-CRITICAL **: 13:07:41.573: gst_iterator_next: assertion 'it != NULL' failed
(edge:2591): GStreamer-CRITICAL **: 13:07:41.573: gst_iterator_free: assertion 'it != NULL' failed
[ WARN:0@9.260] global /root/repos/opencv/modules/videoio/src/cap_gstreamer.cpp (1226) open OpenCV | GStreamer warning: cannot find appsink in manual pipeline
[ WARN:0@9.260] global /root/repos/opencv/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been createdor v2.videoCapture the parameters you use in video-viewer,
dusty-nv
commented
2 years ago I'm not sure why OpenCV says it can't find your appsink element (although I'm not an expert on OpenCV), although this part of your pipeline:
! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12" ! appsinkI think you will want to change to:
! "video/x-raw, width=(int)1920, height=(int)1080, format=(string)NV12" ! appsinkbecause I have special code in jetson-utils that handles NVMM memory, but OpenCV does not.
masip85
commented
2 years ago My opencv c++ software works fine but slow with opencv. My gstreamer pipeline works by itself ( I add a reencoding an mp4 sink.( bad pixel quality) My guess is opencv does not admit NV12 ( with any resolution, same result). Why does video viewer use this? If i want to obtain decoded raw image, is this the best approach?
Regards
El El jue, 31 mar 2022 a las 15:35, Dustin Franklin < @.***> escribió:
Hi @masip85 https://github.com/masip85, my guess is some issue or connectivity on the RTSP source side. It also appears this RTSP source has high resolution of 3072x1728. Maybe you want to try lower resolution at first?
If you just run a normal gstreamer pipeline (without the hardware-accelerated omx decoder), does it work?
— Reply to this email directly, view it on GitHub https://github.com/dusty-nv/jetson-utils/issues/118#issuecomment-1084585863, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCES3QSAMNBEVZG5RCAYFTVCWSY7ANCNFSM5SEYFGHA . You are receiving this because you were mentioned.Message ID: @.***>
dusty-nv
commented
2 years ago Why does video viewer use this?
I use it because the NVIDIA codecs output it and I have a CUDA function that converts NV12 -> RGB.
However, you can add nvvidconv element to your pipeline to convert to the following formats:
format: { (string)I420, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV12, (string)NV16, (string)NV24, (string)P010_10LE, (string)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B }(check gst-inspect-1.0 nvvidconv for more options)
Hello.
I've looking into this issue in forum and github. Normally it happens after an error which is the cause, But,the only error I can see is not retrieving any image. What is happening? Hardware: XAVIER AGX
Same issue here. But disabling that option doesn't solve it
But disabling the NVMM doesn't work. This build2 is with -DENABLE_NVMM=off. :
So,in order to debug I have joined two pipelines into this one adding -vvv
In this way, I am savig empty files. But looks like decoder hasn't retrieved any image either. Does the error happens because of this? (gst-launch-1.0:10288): GStreamer-CRITICAL : 11:18:29.587: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed (gst-launch-1.0:10288): GStreamer-CRITICAL : 11:18:29.587: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed (gst-launch-1.0:10288): GStreamer-CRITICAL : 11:18:29.587: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed (gst-launch-1.0:10288): GStreamer-CRITICAL : 11:18:29.587: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed (gst-launch-1.0:10288): GStreamer-CRITICAL : 11:18:29.587: gst_structure_get_string: assertion 'structure != NULL' failed (gst-launch-1.0:10288): GStreamer-CRITICAL : 11:18:29.587: gst_mini_object_unref: assertion 'mini_object != NULL' failed And what is this?