VS WebRTC Latency

[dannymarosi]

Is there a way to reduce the latency with KVS WebRTC?

I am using kvsWebRTCClientMasterGstreamerSample.c and getting 2sec latency with the following pipeline:

rpicamsrc ! queue ! videoconvert ! video/x-raw,width=1280,height=720,framerate=30/1 ! x264enc bframes=0 speed-preset=veryfast key-int-max=30 bitrate=512 ! video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! appsink sync=TRUE emit-signals=TRUE name=appsink-video

See KVS WebRTC Test Page:

[Sean-Der]

Hey @dannymarosi

What latency do you see from just doing rpicamsrc ! videoconvert ! autovideosink

Do you know what transport is used? It looks like you are in the same LAN, so you should be using host candidates. Can you try disabling TURN in both the test page and client?

The next thing I would check is chrome://webrtc-internals, you can see the additional latency that is being added by the browser (to handle jitter/packet-loss)

[dannymarosi]

Hi @Sean-Der

No video from Master on KVS WebRTC Test Page after changing the pipeline to “rpicamsrc ! videoconvert ! autovideosink” I do not know what transport is used. How can I find out? Yes I am in the same LAN How do I select host candidates? How do I disable TURN? No video form the master when selecting Disabled

[dannymarosi]

here is master side log WebRTC_log.txt

[Sean-Der]

Sorry for the delay @dannymarosi

Would you mind running that pipeline directly from the command line like gst-launch-1.0 rpicamsrc ! videoconvert ! autovideosink

I will work on extending the JS UI to print what transport is being used, I think that would be really helpful for debugging!

I am really surprised when you are on the same LAN that selecting disabled doesn't work. Can the viewer directly access the device via UDP and TCP? Do you have any network policies that could prevent any type of traffic.

[dannymarosi]

Hi @Sean-Der The NAT Traversal Disable selection is working now. We had a SonicWall issue which we believe caused the problem. I still see the 2 second latency with the NAT Traversal Disabled. Running gst-launch-1.0 rpicamsrc ! videoconvert ! autovideosink worked but it take full screen and frame rate is about 1/1 See attached log file.

gst-launch-1.0_log.txt

[Sean-Der]

If you are still seeing 2 seconds of latency when you have everything disabled my guess would be that your media pipeline is introducing the latency!

I don't have a rpicamsrc myself, so can't confirm but looking at the docs you are paying a penalty by doing software encoding. Can you try a pipeline like this?

rpicamsrc keyframe-interval=30
! queue
! h264parse
! video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline
! appsink sync=TRUE emit-signals=TRUE name=appsink-video

[Sean-Der]

I don't know if every camera has hardware encoding, but just from looking at the caps/properties in https://github.com/thaytan/gst-rpicamsrc/blob/master/src/gstrpicamsrc.c picked that up.

[dannymarosi]

Hi @Sean-Der

The delay was completely gone when running that pipeline directly from the command line.

But I was unable to see the video in KVS WebRTC Test page After putting this pipeline in kvsWebRTCClientMasterGstreamerSample.c

        pipeline = gst_parse_launch(
                "rpicamsrc keyframe-interval=30 ! queue ! h264parse ! "
                "video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! appsink sync=TRUE emit-signals=TRUE name=appsink-video",
                &error);

What am I missing?

[chehefen]

hi @dannymarosi, you need to add config-interval=-1 to h264parse to add sps and pps for each idr frame. So far we only observe this behavior on from raspberry pi camera module.

[dannymarosi]

Hi @ chehefen

Adding config-interval=-1 to h264parse did not help.

    pipeline = gst_parse_launch(
                "rpicamsrc keyframe-interval=30 ! queue ! h264parse config-interval=-1 ! "
                "video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! appsink sync=TRUE emit-signals=TRUE name=appsink-video",
                &error);

What else am I missing?

[chehefen]

can you try remove sync=TRUE?

[dannymarosi]

Removing sync=TRUE did not help.

[chehefen]

not sure why it didnt work for you. this worked for me on raspberry with camera module rpicamsrc keyframe-interval=30 ! queue ! h264parse config-interval=-1 ! video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! queue ! appsink emit-signals=TRUE name=appsink-video

[dannymarosi]

Hi @chehefen

My pipeline was missing a queue element between video/x-h264 and appsink element. Once I added the queue element the video from the master start showing.

But we get 1.4 sec latency when running the pipeline on the Pi from kvsWebRTCClientMasterGstreamerSample.c and streaming to PC on the LAN.

pipeline = gst_parse_launch( "rpicamsrc keyframe-interval=30 ! queue ! h264parse config-interval=-1 ! " "video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline ! queue ! appsink emit-signals=TRUE name=appsink-video", &error);

Running your pipeline on the Pi from the command line with gst-launch-1.0 and displaying on the Pi we got almost no latency.

We also get very low latency when streaming KVS WebRTC between 2 browsers on LAN.

Is 1.4sec latency is a reasonable expectation when streaming with KVS WebRTC?

[chehefen]

Hi, I'm running the same code on in my home network on a raspberry pi 3 with camera module and I'm seeing sub-second latency. Can you try to identify if there is any source of latency such as other processes competing for resource on your pi or network? Thanks

[dannymarosi]

@chehefen Hi, I am using pi 2. I was able to reduce the latency to 0.7 seconds by changing log level from default to LOG_LEVEL_ERROR Thanks, Danny Marosi

[dannymarosi]

Hi Anatoli, Setup desired log level: Set up the desired log level. The log levels and corresponding values currently available are:

1. LOG_LEVEL_VERBOSE ---- 1
2. LOG_LEVEL_DEBUG ---- 2
3. LOG_LEVEL_INFO ---- 3
4. LOG_LEVEL_WARN ---- 4
5. LOG_LEVEL_ERROR ---- 5
6. LOG_LEVEL_FATAL ---- 6
7. LOG_LEVEL_SILENT ---- 7 To set a log level, run the following command: export AWS_KVS_LOG_LEVEL = For example: export AWS_KVS_LOG_LEVEL = 2 switches on DEBUG level logs while runnning the samples Note: The default log level is LOG_LEVEL_WARN.

If you are looking into reducing the latency with Pi zero: You can try the pipe line I am using: pipeline = gst_parse_launch( "rpicamsrc ! h264parse config-interval=-1 ! " "video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline,width=640,height=480,framerate=30/1 ! appsink sync=TRUE emit-signals=TRUE name=appsink-video", &error);

From: Anatoli Arkhipenko [mailto:notifications@github.com] Sent: Sunday, July 12, 2020 8:03 AM To: awslabs/amazon-kinesis-video-streams-webrtc-sdk-c Cc: Danny Marosi; Mention Subject: Re: [awslabs/amazon-kinesis-video-streams-webrtc-sdk-c] VS WebRTC Latency (#310)

I was able to reduce the latency to 0.7 seconds by changing log level from default to LOG_LEVEL_ERROR @dannymarosihttps://github.com/dannymarosi Hi, I am fighting similar issues - how did you change the log level? I can't find it in the code of kvsWebRTCClientMasterGstreamerSample.c Thx!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/issues/310#issuecomment-657233725, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO5ZZWQ7XDJJVBY5NY3DYFTR3HGCBANCNFSM4LVJVT6A.

---

This email has been scanned for spam and viruses. Click herehttps://modusCloud.cloud-protect.net/index01.php?mod_id=11&mod_option=logitem&mail_id=1594566177-eWAJrcBW5Kgb&r_address=dmarosi%40casees.com&report=1 to report this email as spam.

[arkhipenko]

Thank you! I found this same documentation in the C version of the repo. Somehow the C++ version does not have this information, and that is where I started....

---

From: dannymarosi notifications@github.com Sent: Monday, July 13, 2020 1:19 PM To: awslabs/amazon-kinesis-video-streams-webrtc-sdk-c amazon-kinesis-video-streams-webrtc-sdk-c@noreply.github.com Cc: Anatoli Arkhipenko arkhipenko@hotmail.com; Comment comment@noreply.github.com Subject: Re: [awslabs/amazon-kinesis-video-streams-webrtc-sdk-c] VS WebRTC Latency (#310)

Hi Anatoli, Setup desired log level: Set up the desired log level. The log levels and corresponding values currently available are:

1. LOG_LEVEL_VERBOSE ---- 1
2. LOG_LEVEL_DEBUG ---- 2
3. LOG_LEVEL_INFO ---- 3
4. LOG_LEVEL_WARN ---- 4
5. LOG_LEVEL_ERROR ---- 5
6. LOG_LEVEL_FATAL ---- 6
7. LOG_LEVEL_SILENT ---- 7 To set a log level, run the following command: export AWS_KVS_LOG_LEVEL = For example: export AWS_KVS_LOG_LEVEL = 2 switches on DEBUG level logs while runnning the samples Note: The default log level is LOG_LEVEL_WARN.

If you are looking into reducing the latency with Pi zero: You can try the pipe line I am using: pipeline = gst_parse_launch( "rpicamsrc ! h264parse config-interval=-1 ! " "video/x-h264,stream-format=byte-stream,alignment=au,profile=baseline,width=640,height=480,framerate=30/1 ! appsink sync=TRUE emit-signals=TRUE name=appsink-video", &error);

From: Anatoli Arkhipenko [mailto:notifications@github.com] Sent: Sunday, July 12, 2020 8:03 AM To: awslabs/amazon-kinesis-video-streams-webrtc-sdk-c Cc: Danny Marosi; Mention Subject: Re: [awslabs/amazon-kinesis-video-streams-webrtc-sdk-c] VS WebRTC Latency (#310)

I was able to reduce the latency to 0.7 seconds by changing log level from default to LOG_LEVEL_ERROR @dannymarosihttps://github.com/dannymarosi Hi, I am fighting similar issues - how did you change the log level? I can't find it in the code of kvsWebRTCClientMasterGstreamerSample.c Thx!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/issues/310#issuecomment-657233725, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO5ZZWQ7XDJJVBY5NY3DYFTR3HGCBANCNFSM4LVJVT6A.

---

This email has been scanned for spam and viruses. Click herehttps://modusCloud.cloud-protect.net/index01.php?mod_id=11&mod_option=logitem&mail_id=1594566177-eWAJrcBW5Kgb&r_address=dmarosi%40casees.com&report=1 to report this email as spam.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/issues/310#issuecomment-657685927, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AACMMTLBT4YKH73JSOJMVODR3M62DANCNFSM4LVJVT6A.