[bug] Too low quality score

holzgeist commented 1 month ago

Describe the bug The quality score reported to the user through server events is too low in some cases.

To Reproduce

Steps to reproduce the behavior:

Configure video capture to be 960x720 (4:3 720 preset)
Connect to a room with certain Android devices that does not support natively capturing that resolution (e.g. Fairphone 5)
Observe connection quality score go down to Good after a few seconds

This even happens on a local dev environment with low ping times/RTT.

Expected behavior Excellent score due to local network being fast enough

Platform information

Flutter version: 3.22.2
Plugin version: 2.1.4
Flutter target OS: Android
Flutter target OS version: 13

Additional context This is tightly related to https://github.com/webrtc-sdk/webrtc/issues/133. However, even with the partially fixing PR applied, the quality score is still too low.

The reason for this issue is that WebRTC Simulcast layer counts (https://github.com/webrtc-sdk/webrtc/blob/7ddfc43fcd2e4ba0548258a76fe88d49b34588c5/video/config/simulcast.cc#L82-L106) do not always match Livekit Flutter SDK layer counts https://github.com/livekit/client-sdk-flutter/blob/main/lib/src/utils.dart#L473-L477. Livekit server then assumes that bad network is the cause for the missing layers

Actual reasons for the mismatch:

this SDK only considers max dimension, WebRTC considers pixel count (16:9 vs 4:3 comes into play too), and
WebRTC potentially resizes video streams before calculating simulcast layers (see issue mentioned above)

cloudwebrtc commented 1 month ago

hey @holzgeist, Another possibility could be related to the DegradationPreference default setting. This PR has set the default value to balanced. Perhaps you can test your patch together with this PR to see if the video quality improves.

https://github.com/livekit/client-sdk-flutter/pull/556

holzgeist commented 1 month ago

Hi @cloudwebrtc Thanks for the hint, I'll check it out. I don't have much hope, because my added log output in scorer.go is pretty unambiguous: "packetScore": 98.72154736490438, "bitrateScore": 100, "layerScore": 65

Full log line with context:

2024-07-15T16:26:36.994+0200    INFO    livekit.pub connectionquality/scorer.go:413 Scorer: {"room": "r1", "roomID": "RM_X6HwsbHncGLj", "participant": "p1", "pID": "PA_iXxtQHmwQwQZ", "remote": false, "trackID": "TR_VC34Ct5b9cB43t", "relayed": false, "mime": "video/vp8", "direction": "up", "expectedDistance": 1, "distanceWeight": 35, "packetScore": 98.72154736490438, "bitrateScore": 100, "layerScore": 65}

holzgeist commented 1 month ago

The PR you mentioned doesn't change anything unfortunately

cloudwebrtc commented 1 month ago

fixed https://github.com/livekit/client-sdk-flutter/releases/tag/v2.2.2

livekit / client-sdk-flutter

[bug] Too low quality score #557