HD Resolution video stucking and frame dropping

[mirosvanek]

Hello.

We use model one broadcaster and many receivers. We need to use HD resolution, however significant frame dropping is presented and many times video stucks for several seconds, even quality significantly decreases and fps decrease. We used screen sharing with fullscreen HD video broadcasted. We also tried to use the same scenario with WebRTC demo at https://apprtc.appspot.com/ (parameters to enable screen capture were changed by console). Video ran quite fluently, just when scene completely changed, some short stucks were presented.

Parameters set: • At demo we changed videoSendBitrate to 100000 (unlimited bandwidth) when connecting peer, mediaStreamConstrants to enable screen capture was when getUserMedia was called • At Licode scenario maxVideoBW to 100000 (unlimited bandwidth)

Suspicions:

For one to one connection (broadcaster -> receiver), at server there was network traffic not higher than 200 kbps (just peeks). For HD stream I would expect 1-2Mbps.

Isn’t there limited stream network connection? Even some default network speed settings for APIs you are using? There are console deprecation reports of express: connect.limit() will be removed in connect 3.0 Isn’t it applied even by some default settings?

WebRTC session description settings:

Parameter maxVideoBW is applied for b=AS: of session description. There's m line defining media and c line defining connection. b line should follow c line by RFC 4566, sample:

... a=ssrc:3632863251 label:a94cdc4e-e4bd-4432-907d-ec4dbffaff43 m=video 1 RTP/SAVPF 100 116 117 c=IN IP4 0.0.0.0 b=AS:100000 a=rtcp:1 IN IP4 0.0.0.0 ...

But Licode puts b line directly after m line, sample:

... a=maxptime:60 m=video 1 RTP/SAVPF 100 116 117 b=AS:100000 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 ...

Can’t this be a network speed limit? Parameter b=AS: is set only at broadcaster side. On receiver’s side, I haven’t found any of these lines. Isn’t it needed also for reciever’s side?

In erizo source code, there is many times SD (640x480) resolution hardcoded. Isn’t Licode tuned for SD resolution?

Unfortunately, scenario above (broadcasting screen shared video) was stucking also for this resolution. Not so significantly, but WebRTC demo with HD resolution had better apereance.

Thank you for answering questions, we appreciate your help.

Best regards, MV

[mirosvanek]

Please, can somebody tell me, if it is possible to stream with Licode on fHD or HD/min 30fps, or there is a problem with codec VP8? Thank you

[lodoyun]

You should be able to stream in full HD in Licode without any problems. However, there are several things you have to take into account at this point:

• You need to change licode_config.js (you got this part right).
• 640x480 is not hardcoded, it is provided in the BasicExample as an example :), you can change this freely, also, the size of the
• The bandwidth limits are enforced, however the actual bit-rate depends on many more factors: for instance, the bandwidth available in the clients can affect the publisher. That is, at this point, in Licode, if you have 1 publisher and 10 subscribers, the publisher tries to adapt to the WORST case so all the subscribers can receive the stream. Also, CPU levels in the publisher can affect the quality.

Although you don't mention recording, I will tell you this is a different context. Recording, at this point, MUST be 640x480.

[mirosvanek]

Thank you for your answer.

• All tests were done on local network with one broadcaster and one receiver, so no slow receiver should have affected broadcaster. However, network speed seemed to be limited at about 200-300kbps regardless licode_config BW value, here we see a problem and we don't know how to move forward
• I would expect, that broadcaster stream is independent on any receivers and there is some kind of stream queue for receivers to have stream quality regarding their connection stability. Maybe this is part for MCU and hopefully receivers can choose quality of streaming.
• Isn't broadcaster's stream on async_events branch independent? Which branch would you recommend to use?
• We did test with and without recording. For recording, we changed the code and passed broadcaster's stream resolution to ExternalOutput to set correct MKV parameters.

[jnoring]

I'd love to see a diff for the change to ExternalOutput to set the mkv parameters.

On Tue, Aug 5, 2014 at 6:08 AM, mirosvanek notifications@github.com wrote:

• All tests were done on local network with one broadcaster and one receiver, so no slow receiver should have affected broadcaster. However, network speed seemed to be limited at about 200-300kbps regardless licode_config BW value, here we see a problem and we don't know how to move forward
• I would expect, that broadcaster stream is independent on any receivers and there is some kind of stream queue for receivers to have stream quality regarding their connection stability. Maybe this is part for MCU and hopefully receivers can choose quality of streaming.
• Isn't broadcaster's stream on async_events branch independent? Which branch would you recommend to use?
• We did test with and without recording. For recording, we changed the code and passed broadcaster's stream resolution to ExternalOutput to set correct MKV parameters.

— Reply to this email directly or view it on GitHub https://github.com/ging/licode/issues/138#issuecomment-51188543.

[lodoyun]

@mirosvanek What application are you using? If it is based on basicExample, please check https://github.com/ging/licode/blob/async_events/extras/basic_example/public/script.js#L70 and change it to the desired bandwidth.

The broadcaster stream is affected by the bandwidth of the subscribers via RTCP feedback, http://tools.ietf.org/html/draft-ietf-rtcweb-rtp-usage-16 has a section about it. The bit-rate control mechanism implemented in the clients (browsers). In the MCU we make sure we forward the RTCP packets so this mechanism can take place. At this point, there is no transcoding in the MCU and there is only one stream quality for each participant. We may introduce some of these features in the future but we are focused on videoconferencing at this point. There are hybrid solutions we can discuss, please create a topic in our list https://groups.google.com/forum/#!forum/lynckia if you have not done it yet and we can continue there.

I'd prefer to focus this issue in making there are no problems with Licode working in HD when there is enough bandwidth and CPU available.

[ivopavlik]

Hello,

@lodoyun, you wrote: "The bandwidth limits are enforced, however the actual bit-rate depends on many more factors: for instance, the bandwidth available in the clients can affect the publisher. That is, at this point, in Licode, if you have 1 publisher and 10 subscribers, the publisher tries to adapt to the WORST case so all the subscribers can receive the stream. Also, CPU levels in the publisher can affect the quality". We have one publisher to many subscribers, where subscribers don't send any stream. Is publisher's stream bandwidth also adapted to the "worst case" - worst client's bandwidth? If so, is it possible to avoid this behavior to provide a stream for client of as highest quality as is possible depending their bandwidth?

Thank you.

[lodoyun]

Yes. The subscribers will affect the video quality of the broadcaster. To avoid that, we would need to change the way we forward RTCP packets. There are some alternatives for this, one is to comment this line to avoid advertising feedback capabilities and see how it works. I have not had time to test this.

[iwuvjhdva]

We have the same issue here, the broadcasted video stream is often interrupted and the video bitrate substantially decreases sometimes. Tried to comment out the line you pointed to, it produces visual artefacts on the video instead of the interruptions. It is less visible on cable connections, more on WiFi and always happens on 4G. Basically I didn't succeed to have the HQ video played totally smoothly on any type of connection, both with the line commented or not. The Licode server bandwidth is stable symmetric 1Gbit.

[iwuvjhdva]

Is there a way to specify minimal possible bitrate i.e. let's say to not to go lower than 500 kbps even if one of the parties has a very bad internet connection?

[iwuvjhdva]

We tried to comment out the line you pointed to above, but bad network on one of the clients still affects the stream quality for other clients.

The testing environment we had: The broadcaster is broadcasting HQ video with two clients launched on two different computers. Then if on one of them we do

iptables -A INPUT -m statistic -p udp --mode random --probability 0.8 -j DROP

The video becomes awful on both clients. Are there any ways to solve that?

[iwuvjhdva]

Have solved the problem by adding minimal bitrate SDP header, line 384 at SdpInfo.cpp:

sdp << "a=fmtp:"<< rtp.payloadType<<" x-google-min-bitrate=1500; x-google-max-bitrate=3000; x-google-max-quantization=56\n";

Could you make an advice the proper way to add this functionality to Licode for me to make a proper pull request, I mean the right place to insert this line and how to make these settings settable via licode_config.js.

THANK YOU!!!

[zevarito]

@iwuvjhdva Hi! They never reply to you? Did this get introduced at the end?

[ivopavlik]

We were using patch of @iwuvjhdva , thanks for it. However, with Chrome 40.0.2214.91 (issue #238) can't broadcast/receive stream with this bitrate control. Can you give us some advices how to use those parameters?

Thanks.

[zevarito]

I used x-google-min-bitrate to avoid 300kbp sending limit on Chrome. However, I wonder why b=AS: param takes no effect in video, it seems to work on audio. I've read that some people make it work by removing nack and set b=AS:, is that a good thing to try?

[wangchengnick]

@iwuvjhdva , where should your patch for HD resolution be exactly add? SdpInfo.cpp in licode/erizo/src/erizo? I had make this change but compiled with errors.

[sciaz]

I've the same problem,has it been sloved yet? @iwuvjhdva @ivopavlik the video is not stable..

[zevarito]

@sciaz did it always happen or just recently?

[kkavod]

Hi all The problem solved for us by moving to Janus Gateway with its Streaming plugin.

[zevarito]

b=AS has no effect on Licode video, it does with audio. If I test it with Test-Bandwidth-Demo[0] using same browser it does limit bandwidth correctly, why it is not working with Licode? Also, b=AS should be after i and c lines.

[0] https://webrtc.github.io/samples/src/content/peerconnection/bandwidth/

[kekkokk]

b=AS does affect the video, but only when at least one subscriber is present.

If you try to join a room and record without any subscriber the bandwidth will remain = 300kbps and the stream would be full of artifacts.

If you try to self-subscribe then everything works as expected.

To limit the min-bandwidth you have to update your video specs (ex. localStream.updateConfiguration({minVideoBW: 1000, Sdp: true}). Then the mechanism of feedback will be interrupted for those subscribers whose have a bandwidth < 1000kbps, allowing the publisher to adjust its quality to the worst subscriber which have >= 1000kbps.

Without a proper feedback termination mechanism on Licode side this can be very useful but also very risky. I don't know how it will react if you raise the minVideoBW a lot and no one of the subscriber has the proper bandwidth.

Need to test

[zevarito]

@kekkokk thanks for the clarification, indeed it works like that on web, couldn't confirm the same for IOS, video started to go up after one subscriber but never hit the limit and decrease to 300kbps almost immediately, audio bandwidth didn't present any change.

[kekkokk]

I'll start working on iOS in January I think, let's keep in touch!

[thehappycoder]

@iwuvjhdva Licode doesn't seem to support any of x-google-* flags. How could it have helped?

[kekkokk]

@thehappycoder If you set that parameter in your local description, you are telling your browser to not fall below that bandwidth, so it ignores the incoming rembs lower than that. Licode does not matter in this case

[iwuvjhdva]

We have solved all the broadcasting issues by switching to Janus.

On Thu, Sep 7, 2017 at 9:11 AM, Francesco Durighetto < notifications@github.com> wrote:

@thehappycoder https://github.com/thehappycoder If you set that parameter in your local description, you are telling your browser to not fall below that bandwidth, so it ignores the incoming rembs lower than that. Licode does not matter in this case

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lynckia/licode/issues/138#issuecomment-327708852, or mute the thread https://github.com/notifications/unsubscribe-auth/AALbCtFSkeio2D1SG0jPAGgHKKWxNwYxks5sf5c1gaJpZM4CIu41 .

-- Kind regards, Konstantin Alexandrov