Buffer behaves abnormally in Window-Sink when playing rtsps (TLS/SSL)

[YoungjaeDev]

hello. Unlike the existing rtsp, there is a situation where I have to receive input with rtsps with tsl/ssl applied.

First of all, to explain the situation, currently there is no need for certification related to Cert, and it is enough to simply receive it through rtsps. Currently, the tls-validation-flags property of Gstreamer rtspsrc is set to validate-all (0x0000007f) – G_TLS_CERTIFICATE_VALIDATE_ALL, so if I use it with a different value that does not require Cert authentication, the error below will disappear.

https://thiblahute.github.io/GStreamer-doc/rtspclientsink-1.0/rtspclientsink.html#GstRTSPClientSink:tls-validation-flags

0:00:00.040470735  9597 0x55b3a66b6640 ERROR                default gstrtspconnection.c:1046:gst_rtsp_connection_connect_with_response: failed to connect: Unacceptable TLS certificate

---

However, no errors occur, but there are issues that occur when running in gst-launch in terminal log

1. Still waiting in the terminal with no video coming out

   gst-launch-1.0 rtspsrc location="rtsps://192.168.0.40:30322/stream-eo" tls-validation-flags=0 latency=100 ! decodebin ! autovideosink
   Setting pipeline to PAUSED ...
   Pipeline is live and does not need PREROLL ...
   Progress: (open) Opening Stream
   Progress: (connect) Connecting to rtsps://192.168.0.40:30322/stream-eo
   Progress: (open) Retrieving server options
   Progress: (open) Retrieving media info
   Progress: (request) SETUP stream 0
   Progress: (request) SETUP stream 0
   Progress: (request) SETUP stream 0
   Progress: (open) Opened Stream
   Setting pipeline to PLAYING ...
   New clock: GstSystemClock
   Progress: (request) Sending PLAY request
   Progress: (request) Sending PLAY request
   Progress: (request) Sent PLAY request
   WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Delayed linking failed.
   Additional debug info:
   ./grammar.y(506): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0:
   failed delayed linking some pad of GstDecodeBin named decodebin0 to some pad of GstAutoVideoSink named autovideosink0

2. The video comes out, but there are too many skip frames.

   gst-launch-1.0 rtspsrc location="rtsps://192.168.0.40:30322/stream-eo" tls-validation-flags=4 latency=100 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink
   Setting pipeline to PAUSED ...
   Pipeline is live and does not need PREROLL ...
   Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
   Progress: (open) Opening Stream
   Progress: (connect) Connecting to rtsps://192.168.0.40:30322/stream-eo
   Progress: (open) Retrieving server options
   Progress: (open) Retrieving media info
   Progress: (request) SETUP stream 0
   Progress: (request) SETUP stream 0
   Progress: (request) SETUP stream 0
   Progress: (open) Opened Stream
   Setting pipeline to PLAYING ...
   New clock: GstSystemClock
   Progress: (request) Sending PLAY request
   Progress: (request) Sending PLAY request
   Progress: (request) Sent PLAY request
   WARNING: from element /GstPipeline:pipeline0/GstEglGlesSink:eglglessink0: A lot of buffers are being dropped.
   Additional debug info:
   gstbasesink.c(3003): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstEglGlesSink:eglglessink0:
   There may be a timestamping problem, or this computer is too slow.

3. When the Player code is executed after applying the changes below in DslSourceBintr.cpp in DSL, I will share it by DM, but there are cases where many frames are received at once because Frames come in sequentially and are buffered once.

...
std::wstring eo_rtsp_url = L"rtsps://192.168.0.40:30322/stream-eo";
std::wstring ir_rtsp_url = L"rtsps://192.168.0.40:30322/stream-ir";
...

int main(int argc, char** argv)
{  
    DslReturnType retval;

    // # Since we're not using args, we can Let DSL initialize GST on first call    
    while(true){    

        // # For each camera, create a new RTSP Source for the specific RTSP URI    
        retval = dsl_source_rtsp_new(L"rtsp-source", ir_rtsp_url.c_str(), DSL_RTP_ALL,     
            false, 0, 100, 2);
        if (retval != DSL_RESULT_SUCCESS)    
            return retval;

        retval = dsl_source_uri_new(L"video-source", video_uri.c_str(), false, false, 0);
        if (retval != DSL_RESULT_SUCCESS)    
            return retval;

        // # New Overlay Sink, 0 x/y offsets and same dimensions as Tiled Display    
        retval = dsl_sink_window_new(L"window-sink", 0, 0, WINDOW_WIDTH, WINDOW_HEIGHT);
        if (retval != DSL_RESULT_SUCCESS) break;

        retval = dsl_player_new(L"player", L"rtsp-source", L"window-sink");
        if (retval != DSL_RESULT_SUCCESS) break;

        // # Add the XWindow event handler functions defined above    
        retval = dsl_player_xwindow_key_event_handler_add(L"player", xwindow_key_event_handler, nullptr);    
        if (retval != DSL_RESULT_SUCCESS) break;    

        // # Play the player    
        retval = dsl_player_play(L"player");
        if (retval != DSL_RESULT_SUCCESS) break;

        dsl_main_loop_run();
        retval = DSL_RESULT_SUCCESS;
        break;
    }
}

Current source is in RGB-3840x1080 format To summarize, I currently simply change to rtsps and change the tls-validation-flags, but the above issues occur.

[rjhowell44]

@youngjae-avikus I can add services to get/set the tls-validation-flags... I'll try and review the other issue tonight.

[YoungjaeDev]

@rjhowell44 It doesn't seem like a server problem, seeing that URLs are played normally when called from OpenCV to VideoCapture.

[rjhowell44]

@youngjae-avikus are testing on Jetson or dGPU? If Jetson can you try with overlay sink instead of window sink? https://github.com/prominenceai/deepstream-services-library/blob/proc_mode/docs/api-sink.md#dsl_sink_overlay_new

This launch command looks correct to me, although you don't need the parser with nvv4l2decoder

gst-launch-1.0 rtspsrc location="rtsps://192.168.0.40:30322/stream-eo" tls-validation-flags=4 latency=100 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

Also, it might be worth trying to set the Window Sink's sync behavior to false https://github.com/prominenceai/deepstream-services-library/blob/proc_mode/docs/api-sink.md#dsl_sink_sync_enabled_set

We may need to get NVIDIA involved

[YoungjaeDev]

@rjhowell44 are testing on Jetson or dGPU?

This project is only for dGPUs.

[YoungjaeDev]

@rjhowell44

Even if sink=false, a similar phenomenon still occurs. But can you see that the other parts are normal and the sink is a little unstable?

// # New Overlay Sink, 0 x/y offsets and same dimensions as Tiled Display    
retval = dsl_sink_window_new(L"window-sink", 0, 0, WINDOW_WIDTH, WINDOW_HEIGHT);
if (retval != DSL_RESULT_SUCCESS) break;

retval = dsl_sink_sync_enabled_set(L"window-sink", false);
if (retval != DSL_RESULT_SUCCESS) break;

retval = dsl_player_new(L"player", L"rtsp-source", L"window-sink");
if (retval != DSL_RESULT_SUCCESS) break;

[YoungjaeDev]

@rjhowell44

When GST_DEBUG=3, the warnings below appear. Is there a problem?

0:00:00.052740294  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.052754114  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:341:HandleStateChanged:  : GST_STATE_PAUSED => GST_STATE_PLAYING
0:00:00.052765081  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.076739703  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.115534771  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.119781626  3880 0x55cc4f69d980 INFO                     DSL DslSourceBintr.cpp:2955:HandleSelectStream:  : Media = 'video' for RtspSourceBitnr 'rtsp-source'
0:00:00.119794981  3880 0x55cc4f69d980 INFO                     DSL DslSourceBintr.cpp:2957:HandleSelectStream:  : Encoding = 'H264' for RtspSourceBitnr 'rtsp-source'
0:00:00.122010192  3880 0x55cc4f69d980 WARN                  udpsrc gstudpsrc.c:1445:gst_udpsrc_open:<udpsrc0> warning: Could not create a buffer of requested 524288 bytes (Operation not permitted). Need net.admin privilege?
0:00:00.122029157  3880 0x55cc4f69d980 WARN                  udpsrc gstudpsrc.c:1455:gst_udpsrc_open:<udpsrc0> have udp buffer of 212992 bytes while 524288 were requested
0:00:00.122127031  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.127896318  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.133713373  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:00.136816954  3880 0x55cc4f69ba70 INFO                     DSL DslSourceBintr.cpp:3303:SetCurrentState:  : Changing state from NULL to PLAYING for RtspSourceBintr 'rtsp-source'
0:00:01.140046999  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:01.140066429  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:01.147903248  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: progress
0:00:01.154309506  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:303:HandleBusWatchMessage:  : Message type:: stream-status
0:00:01.154390225  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:303:HandleBusWatchMessage:  : Message type:: stream-status
0:00:01.154446395  3880 0x55cc4f69e360 INFO                     DSL DslSourceBintr.cpp:3065:HandleSourceElementOnPadAdded:  : Caps structs name application/x-rtp
0:00:01.154463326  3880 0x55cc4f69e360 INFO                     DSL DslSourceBintr.cpp:3066:HandleSourceElementOnPadAdded:  : Media = 'video' for RtspSourceBitnr 'rtsp-source'
0:00:01.154492248  3880 0x55cc4f69e360 INFO                     DSL DslSourceBintr.cpp:3090:HandleSourceElementOnPadAdded:  : rtspsrc element linked for RtspSourceBintr 'rtsp-source'
0:00:01.154500534  3880 0x55cc4f69e360 INFO                     DSL DslSourceBintr.cpp:3102:HandleSourceElementOnPadAdded:  : Frame width = 0, height = 0
0:00:01.154507165  3880 0x55cc4f69e360 INFO                     DSL DslSourceBintr.cpp:3103:HandleSourceElementOnPadAdded:  : FPS numerator = 0, denominator = 0
0:00:01.154599755  3880 0x55cc4f69e000 FIXME               basesink gstbasesink.c:3246:gst_base_sink_default_event:<window-sink-nveglglessink> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:01.154630581  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:325:HandleBusWatchMessage:  : Unhandled message type:: stream-start
0:00:01.351460944  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:303:HandleBusWatchMessage:  : Message type:: element
0:00:01.436332319  3880 0x55cc4f69daa0 WARN            v4l2videodec gstv4l2videodec.c:1836:gst_v4l2_video_dec_decide_allocation:<rtsp-source-nvv4l2decoder> Duration invalid, not setting latency
0:00:01.436599169  3880 0x55cc4f69daa0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1082:gst_v4l2_buffer_pool_start:<rtsp-source-nvv4l2decoder:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:01.436675925  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:303:HandleBusWatchMessage:  : Message type:: stream-status
0:00:01.436785174  3880 0x55cc4f69ba70 INFO                     DSL DslPipelineStateMgr.cpp:303:HandleBusWatchMessage:  : Message type:: stream-status
0:00:02.857189916  3880 0x7f867c007760 WARN          v4l2bufferpool gstv4l2bufferpool.c:1533:gst_v4l2_buffer_pool_dqbuf:<rtsp-source-nvv4l2decoder:pool:src> Driver should never set v4l2_buffer.field to ANY

[YoungjaeDev]

The WARNING above appears even when testing with an RTSP camera.

[rjhowell44]

We're going to need to get some help on this. Can you capture a couple of logs, one using RTSPS TLS/SSL, the other RTSP setting the log level to export GST_DEBUG=4

[rjhowell44]

reviewing these links https://www.google.com/search?q=gstbasesink.c(3003)%3A+gst_base_sink_is_too_late+()%3A+%2FGstPipeline%3Apipeline0%2FGstEglGlesSink%3Aeglglessink0%3A+There+may+be+a+timestamping+problem%2C+or+this+computer+is+too+slow.&oq=gstbasesink.c(3003)%3A+gst_base_sink_is_too_late+()%3A+%2FGstPipeline%3Apipeline0%2FGstEglGlesSink%3Aeglglessink0%3A+There+may+be+a+timestamping+problem%2C+or+this+computer+is+too+slow.&aqs=chrome..69i57.1727j0j7&sourceid=chrome&ie=UTF-8

[rjhowell44]

@youngjae-avikus new issue for the TLS certificate validation flags https://github.com/prominenceai/deepstream-services-library/issues/992

The work has been completed and merged into the v0.27.alpha branch

[rjhowell44]

Once you get the logs for both cases, please also check for cpu utilization as well - both. Provide me with your dGPU info and NVIDIA driver version and I'll open the topic with NVIDIA. If we can't get the help we need I'll try stack-overflow and the RTSP-Source git-hub site as well.

[YoungjaeDev]

We're going to need to get some help on this. Can you capture a couple of logs, one using RTSPS TLS/SSL, the other RTSP setting the log level to export GST_DEBUG=4

I just sent you a DM

[YoungjaeDev]

Once you get the logs for both cases, please also check for cpu utilization as well - both. Provide me with your dGPU info and NVIDIA driver version and I'll open the topic with NVIDIA. If we can't get the help we need I'll try stack-overflow and the RTSP-Source git-hub site as well.

When comparing the normal situation and the non-normal situation, there seems to be no significant difference in the cpu util. GPU: GPU 0: NVIDIA GeForce GTX 1660 SUPER (UUID: GPU-8aeb9d0f-302f-eca7-a7cc-4470ae7a9eac NVIDIA driver version: 12.0

[rjhowell44]

@youngjae-avikus I posted this yesterday.. no response just yet. https://forums.developer.nvidia.com/t/using-rtsps-tls-ssl-vs-rtsp-results-in-performance-rendering-issues/257585

[YoungjaeDev]

@youngjae-avikus I posted this yesterday.. no response just yet. https://forums.developer.nvidia.com/t/using-rtsps-tls-ssl-vs-rtsp-results-in-performance-rendering-issues/257585

Thank you for letting me follow-up

[YoungjaeDev]

@rjhowell44 The gst-launch below works without interruption, but what difference does it make?

gst-launch-1.0 -v rtspsrc location=rtsp://192.168.0.40:30554/stream-eo?token=avikus ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

It doesn't seem to be a problem with the current rtsp, rtsps, and it seems that the component doesn't fit based on the current player.

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

[YoungjaeDev]

The current video is stuttering and it is a png made by doxygen in the pipeline that is a problem.

[YoungjaeDev]

The pipeline below replacing autovideosink with nveglglessink also works!

gst-launch-1.0 -v rtspsrc location=rtsp://192.168.0.40:30554/stream-eo?token=avikus ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

[rjhowell44]

@youngjae-avikus sorry, but I'm really struggling to understand what you're trying to point out.

re: component doesn't fit based on the current player.

Can you please explain what this means?? component doesn't fit???

Also re: The gst-launch below works without interruption, but what difference does it make?

which command works, the first one, the second one, or both?? Sorry but it is unclear what you are describing.

and there seems to be nothing wrong in the pipeline image provided??

[YoungjaeDev]

sorry, but I'm really struggling to understand what you're trying to point out.

sorry. I'm not familiar with English, so I think the delivery was inexperienced.

re: component doesn't fit based on the current player. Can you please explain what this means?? component doesn't fit???

Oh, I said above that I succeeded in executing with the pipeline below.

gst-launch-1.0 -v rtspsrc location=rtsp://192.168.0.40:30554/stream-eo?token=avikus ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

And player refers to dsl_player_play used by dsl (rtsp-source -> window-sink). However, when this code is executed, There was a phenomenon that the video I shared before stopped every moment

Also re: The gst-launch below works without interruption, but what difference does it make?

gst-launch-1.0 -v rtspsrc location=rtsp://192.168.0.40:30554/stream-eo?token=avikus ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

gst-launch-1.0 -v rtspsrc location=rtsp://192.168.0.40:30554/stream-eo?token=avikus ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

All 3 commands above are working fine. I asked for help because there was something you and I were missing in the Dsl.

[rjhowell44]

@youngjae-avikus are you now saying that RTSP in DSL does not work?

[YoungjaeDev]

@youngjae-avikus are you now saying that RTSP in DSL does not work?

Yes It is. https://forums.developer.nvidia.com/t/using-rtsps-tls-ssl-vs-rtsp-results-in-performance-rendering-issues/257585 When running DSL, like the video left in the question here, it buffers quickly and then stops.

Sorry for the confusion I caused in the beginning, but the conclusion was that the above phenomenon appeared when both rtsp/rtsps were played. At first, I thought the self-test media (webrtc) server built by my company was wrong, but since it runs well with opencv or gst-launch as above, I don't know what the alternative cause is.

[rjhowell44]

I'm so confused I though we were able to reproduce the issue with the command line as we showed NVIDIA? Now you are saying that the command line works, and DSL is failing with regular RTSP?

You've never had a problem with regular RTSP in DSL before correct? what is different now?

[YoungjaeDev]

I'm so confused I though we were able to reproduce the issue with the command line as we showed NVIDIA? Now you are saying that the command line works, and DSL is failing with regular RTSP?

You've never had a problem with regular RTSP in DSL before correct? what is different now?

sorry for the confusion I probably need to communicate with the developer who manages the media server again.

I'm so confused I though we were able to reproduce the issue with the command line as we showed NVIDIA?

you're right. Looking at it again, it seems that Play works well now with that command.

Now you are saying that the command line works, and DSL is failing with regular RTSP?

Unfortunately, that's right. So, the dot I uploaded at the top was extracted from DSL, and the upload from discord was from gst-launch, but I was wondering if there was anything you could infer from this.

You've never had a problem with regular RTSP in DSL before correct? what is different now?

Oddly enough, getting it from the camera(rtsp-jpeg) directly to the IP input works fine. I develop large ships and motor-boat by dividing them, which corresponds to small ships, and problems arise in large ships that operate based on media servers. Need to compare DSL commits that I use once?

[rjhowell44]

@youngjae-avikus thanks for the above, I think I'm getting it now.

First a comment about the pipeline images. Gstreamer creates the graph for the pipeline when it changes state. NULL -> READY, READY->PLAYING, PLAYING->PAUSED, etc. The graphs you shared where from the last change which is READY->PLAYING, i.e. when the Pipeline first transitions to playing.

Now, consider that the DSL RTSP Source is designed to support H264, H265, and now JPEG. To support this, the Source delays creating and connecting all the components (parser and depay) until after the Pipeline is playing and the stream format can be read. So when you dump an image in DSL, unfortunately, you will see the RTSP Source in an incomplete state as in the image you posted above.

When you test on the command line, you select which parser and depay and everything is linked before playing. In this case the image shows the RTSP Source in a complete state.

---

Next steps.

So it sound like it has more to do with the nature of the stream coming from the media servers. Is it less steady/consistent than the RTSP IP stream?

Can you please try the following and report in detail, with as much clarity as possible.

1. In DSL, can you set the latency on the RTSP Source to a full 2 seconds (2000) and try again. The default for the rtspsrc plugin is 1000. I believe you may be using 100 in your code?
2. Can you try setting the buffer-out-frame-rate to the expected rate and see if that smooths out the rendering. See here
3. I'm going to make a build for you to try - in v0.26.alpha, I made some changes because of my NVIDIA discussion here. I'm going to hack in some changes so that the source is exactly like the command line.

[rjhowell44]

@youngjae-avikus for number 1. above, can you also set the sync enabled to false for the Window Sink

[YoungjaeDev]

@rjhowell44 Yes, even if you combine sync-disable with number 1 you said, the problem still occurs

// # For each camera, create a new RTSP Source for the specific RTSP URI    
retval = dsl_source_rtsp_new(L"rtsp-source", eo_rtsp_url.c_str(), DSL_RTP_ALL,     
    false, 0, 2000, 0);
if (retval != DSL_RESULT_SUCCESS)    
    return retval;

retval = dsl_source_uri_new(L"video-source", video_uri.c_str(), false, false, 0);
if (retval != DSL_RESULT_SUCCESS)    
    return retval;

// # New Overlay Sink, 0 x/y offsets and same dimensions as Tiled Display    
retval = dsl_sink_window_new(L"window-sink", 0, 0, WINDOW_WIDTH, WINDOW_HEIGHT);
if (retval != DSL_RESULT_SUCCESS) break;

retval = dsl_sink_sync_enabled_set(L"window-sink", false);
if (retval != DSL_RESULT_SUCCESS) break;

// retval = dsl_sink_file_new(L"file-sink", L"./output.mp4", DSL_CODEC_H264, DSL_CONTAINER_MP4, 4000000, 0);
// if (retval != DSL_RESULT_SUCCESS) break;

const wchar_t* pipeline_component_names[] = 
{
    L"rtsp-source",
    L"window-sink",
    NULL
};

The upstream commit I'm currently using is:

[YoungjaeDev]

@rjhowell44

Now, consider that the DSL RTSP Source is designed to support H264, H265, and now JPEG. To support this, the Source delays creating and connecting all the components (parser and depay) until after the Pipeline is playing and the stream format can be read. So when you dump an image in DSL, unfortunately, you will see the RTSP Source in an incomplete state as in the image you posted above.

But in the end, I can see that the depay parser is properly connected, so the pipeline of DSL doesn't seem to be a problem...

[YoungjaeDev]

@rjhowell44

Oh, change it to egglesink, not autovideosink, and enter the following, the phenomenon is reproduced.

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink sync=0

But strangely, if I set sync to 1 for nveglglessink, the frame skips a lot, and it comes out as an image rather than a video.

Additionally, it doesn't seem to have anything to do with rtspsrc property latency.

[rjhowell44]

@youngjae-avikus I think we're getting somewhere, but I what to make that I have this correct.

With the command line below - using the nveglglessink the phenomenon is reproduced.

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink sync=0

But if you change the sink to autovideosink as follows everything looks good

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

Please confirm... make sure.... and I''ll share this information with Fiona on the NVIDIA thread

Also, can you please try the following (number 2 from the items above)

2. Can you try setting the buffer-out-frame-rate to the expected rate and see if that smooths out the rendering. See here

[YoungjaeDev]

@rjhowell44

fine, latency=1000 (default = 2000) is also ok!

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

bad, skip_frame

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

bad, A phenomenon in which the video moves quickly and then stops, such as in a DSL program

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink sync=0

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink sync=0

Ah, again, there doesn't seem to be much difference between autovideosink and nveglglessink. However, even if I run DSL with the option corresponding to fine, bad, A phenomenon in which the video moves quickly and then stops, such as in a DSL program this phenomenon repeats....

// # For each camera, create a new RTSP Source for the specific RTSP URI    
retval = dsl_source_rtsp_new(L"rtsp-source", eo_rtsps_url.c_str(), DSL_RTP_ALL,     
    false, 0, 2000, 2);
if (retval != DSL_RESULT_SUCCESS)    
    return retval;

retval = dsl_source_rtsp_tls_validation_flags_set(L"rtsp-source", 0x04);
if (retval != DSL_RESULT_SUCCESS)    
    return retval;

// # New Overlay Sink, 0 x/y offsets and same dimensions as Tiled Display    
retval = dsl_sink_window_new(L"window-sink", 0, 0, WINDOW_WIDTH, WINDOW_HEIGHT);
if (retval != DSL_RESULT_SUCCESS) break;

retval = dsl_sink_sync_enabled_set(L"window-sink", false);
if (retval != DSL_RESULT_SUCCESS) break;

const wchar_t* pipeline_component_names[] = 
{
    L"rtsp-source",
    L"window-sink",
    NULL
};

retval = dsl_sink_sync_enabled_set(L"window-sink", false); or retval = dsl_sink_sync_enabled_set(L"window-sink", true); had same phenomenon

[YoungjaeDev]

Also, can you please try the following (number 2 from the items above)

2. Can you try setting the buffer-out-frame-rate to the expected rate and see if that smooths out the rendering. See here

I will contact you as soon as I try

[YoungjaeDev]

@youngjae-avikus I think we're getting somewhere, but I what to make that I have this correct.

With the command line below - using the nveglglessink the phenomenon is reproduced.

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus latency=200 tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! nveglglessink sync=0

But if you change the sink to autovideosink as follows everything looks good

gst-launch-1.0 -v rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! autovideosink

Please confirm... make sure.... and I''ll share this information with Fiona on the NVIDIA thread

Also, can you please try the following (number 2 from the items above) 2. Can you try setting the buffer-out-frame-rate to the expected rate and see if that smooths out the rendering. See here

Sorry, the fps coming out of the media server is not constant, so I'm not sure how many. Even if you set the FPS to a random value, the same phenomenon repeats.

[rjhowell44]

@youngjae-avikus re: However, even if I run DSL with the option corresponding to fine, bad, A phenomenon in which the video moves quickly and then stops, such as in a DSL program this phenomenon repeats....

Is this true if drop-on-latency is false?

[YoungjaeDev]

@rjhowell44

@youngjae-avikus re: However, even if I run DSL with the option corresponding to fine, bad, A phenomenon in which the video moves quickly and then stops, such as in a DSL program this phenomenon repeats....

Is this true if drop-on-latency is false?

I did this by editing

[rjhowell44]

@youngjae-avikus can you try the working command line without the parser

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=a
vikus tls-validation-flags=4 ! rtph264depay ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

I would like to see if this causes the phenomena

[YoungjaeDev]

@youngjae-avikus can you try the working command line without the parser

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=a
vikus tls-validation-flags=4 ! rtph264depay ! nvv4l2decoder ! nvvideoconvert ! nveglglessink

I would like to see if this causes the phenomena

It's fine

gst-launch-1.0 rtspsrc location=rtsps://192.168.0.40:30322/stream-eo?token=avikus tls-validation-flags=4 ! rtph264depay ! nvv4l2decoder ! nvvideoconvert ! nveglglessink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsps://192.168.0.40:30322/stream-eo?token=avikus
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:14.004519971
Setting pipeline to NULL ...