search

selkies-project / selkies-gstreamer

Open-Source Low-Latency Accelerated Linux WebRTC HTML5 Remote Desktop Streaming Platform for Self-Hosting, Containers, Kubernetes, or Cloud/HPC
https://selkies-project.github.io/selkies-gstreamer/
Mozilla Public License 2.0
381 stars 50 forks source link

Selkies stutters during congestion #78

Closed radhikamathuria25 closed 7 months ago

radhikamathuria25 commented 1 year ago

Hi, I am using one of the desktops and am experiencing keymap issues. If I start firefox, I am unable to type anything in the search bar. Restarting the container fixes that, but I am still unable to use the caps lock key.

Thanks in advance.

ehfd commented 1 year ago

Hmm, weird issue... Might be something with Selkies. Does it work with noVNC enabled?

ehfd commented 1 year ago

I am going to revamp the repository soon; I have also had some issues myself.

ehfd commented 1 year ago

Changed Xfce to KDE. Please see if your issue is resolved.

ehfd commented 1 year ago

@radhikamathuria25 This is a web UI issue. Something in Chromium and Firefox 109 is breaking things for some reason.

ehfd commented 1 year ago

image

Edge 109: VideoDecoder: is empty!

ehfd commented 1 year ago

Same phenomenon in v1.3.1, which I was using for four months without this issue.

ehfd commented 1 year ago

Need to check that this is not my computer.

ehfd commented 1 year ago

I did a reboot and checked both Edge on Windows, Firefox on Windows (v1.3.5 since it doesn't work on v1.3.1), and Chrome on Android with v1.3.1. Something in the Web Browser API changed.

ehfd commented 1 year ago

image

Old, unupgraded version of Edge on v1.3.1, 108.0.1462.54.

callTx commented 1 year ago

I didn't have no pay attention for this issue yet. Is it an issue related to Gstreamer WebRTC?

ehfd commented 1 year ago

I didn't have no pay attention for this issue yet. Is it an issue related to Gstreamer WebRTC?

YES.

ehfd commented 1 year ago

image

v1.3.5 on Edge 108.0.1462.54.

ehfd commented 1 year ago

image

v1.3.5 on Edge 109.0.1518.55.

Same session, host, client as the above.

mixxen commented 1 year ago

Same issue but the cursor is visible and screen is black:

image

Chrome Version 109.0.5414.74 (Official Build) (64-bit)

ehfd commented 1 year ago

I see: No error from Selkies, but if I refresh the browser the interface is not reachable. And that the DataChannel (cursor clicks) do get sent while the video is stuck (can be found if you restart selkies-gstreamer).

vamman commented 1 year ago

Confirmed issue here. Most recent Chrome browser.

ehfd commented 1 year ago

image Firefox 109.0. Windows. Version 1.3.6.

ehfd commented 1 year ago

Please try again on v1.3.6 (but I don't know if I made a fix or not).

Video decoder: should still be empty.

ehfd commented 1 year ago

Still the freeze comes to me after about an hour...

ehfd commented 1 year ago

Still the freeze comes to me after about an hour...

To everyone: Disable Sleeping Tabs and try again.

danisla commented 1 year ago

Is this only with relay type connections? If so, it could be something with the TURN server closing the connection, expired credentials, etc.

ehfd commented 1 year ago

Is this only with relay type connections? If so, it could be something with the TURN server closing the connection, expired credentials, etc.

Unknown. But something changed with Chromium 109.

ehfd commented 1 year ago

@danisla It could indeed be something with the sleeping tabs function.

ehfd commented 1 year ago

My issue was resolved by turning off sleeping tabs. Please leave your findings after doing this if you are affected.

mixxen commented 1 year ago

I've tried v1.3.7 on Chrome 109.0.541.74 with Auto Discardable disabled and still see a blank screen with just the cursor.

ehfd commented 1 year ago

I've tried v1.3.7 on Chrome 109.0.541.74 with Auto Discardable disabled and still see a blank screen with just the cursor.

Does it connect? Or does it just throw errors?

mixxen commented 1 year ago

Same issue as before. It connects but screen is black, and the "Video decoder:" is empty.

ehfd commented 1 year ago

@mixxen Did it work before the browser was upgraded?

ehfd commented 1 year ago

@mixxen I see your phenomenon, but if I refresh the webpage, the screen comes back to me.

ehfd commented 1 year ago

Same issue as before. It connects but the screen is black, and the "Video decoder:" is empty.

I need your stdout/stderr logs otherwise. If you are using the container, it's in /tmp.

mixxen commented 1 year ago 
user@5b9341c08e26:/tmp$ cat entrypoint-stdout---supervisor-veon_rdj.log 
 * Starting system message bus dbus
   ...done.
Creating directory NVIDIA-Linux-x86_64-515.65.01
Verifying archive integrity... OK
Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 515.65.01................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

WARNING: You specified the '--no-kernel-modules' command line option, nvidia-installer will not install any kernel modules as part of this driver installation, and it will not remove existing NVIDIA kernel modules not part of an earlier NVIDIA driver installation.  Please ensure that NVIDIA kernel modules matching this driver version are installed separately.

ERROR: Unable to create '/lib/firmware/nvidia/515.65.01/gsp.bin' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-smi' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-debugdump' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libcuda.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-cuda-mps-control' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-cuda-mps-server' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-persistenced' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-compiler.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-tls.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvoptix.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-ngx.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvcuvid.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-encode.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.515.65.01' for copying (Read-only file system)

ERROR: Failed to execute `/usr/bin/systemctl daemon-reload`: System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

WARNING: The installed file '/lib/firmware/nvidia/515.65.01/gsp.bin' has permissions 0644, but it was installed with permissions 0444.

WARNING: Unable to locate/open X configuration file.

Package xorg-server was not found in the pkg-config search path.
Perhaps you should add the directory containing `xorg-server.pc'
to the PKG_CONFIG_PATH environment variable
No package 'xorg-server' found
Option "ProbeAllGpus" "False" added to Screen "Screen0".
Option "BaseMosaic" "False" added to Screen "Screen0".
Option "AllowEmptyInitialConfiguration" "True" added to Screen "Screen0".
New X configuration file written to '/etc/X11/xorg.conf'

Waiting for X socket
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

X.Org X Server 1.20.13
X Protocol Version 11, Revision 0
Build Operating System: linux Ubuntu
Current Operating System: Linux 5b9341c08e26 4.18.0-372.19.1.el8_6.x86_64 #1 SMP Mon Jul 18 11:14:02 EDT 2022 x86_64
Kernel command line: BOOT_IMAGE=(hd0,gpt2)/vmlinuz-4.18.0-372.19.1.el8_6.x86_64 root=/dev/mapper/rhel-root ro resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet fips=1 boot=UUID=64eeaa30-2659-4a54-98ad-02c72b68519d audit=1 audit_backlog_limit=8192 pti=on vsyscall=none page_poison=1 slub_debug=P rd.driver.blacklist=nouveau
Build Date: 07 December 2022  01:02:34PM
xorg-server 2:1.20.13-1ubuntu1~20.04.5 (For technical support please see http://www.ubuntu.com/support) 
Current version of pixman: 0.38.4
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/home/user/.local/share/xorg/Xorg.0.log", Time: Mon Jan 30 20:09:04 2023
(==) Using config file: "/etc/X11/xorg.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
X socket is ready
Session Running. Press [Return] to exit.
/etc/entrypoint.sh: line 134: startkde: command not found

Looks like "startkde" is not found. I'll look into it.

mixxen commented 1 year ago

I rebuilt the "docker-nvidia-glx-desktop" container with the latest updates and that fixed the startkde not found error. Here is the the output from the /tmp/entrypoint...log and /tmp/selkies-gstreamer-stdout...log

user@d9cca5990280:/tmp$ cat entrypoint-stdout---supervisor-g34o1pqa.log 
 * Starting system message bus dbus
   ...done.
Creating directory NVIDIA-Linux-x86_64-515.65.01
Verifying archive integrity... OK
Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 515.65.01................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

WARNING: You specified the '--no-kernel-modules' command line option, nvidia-installer will not install any kernel modules as part of this driver installation, and it will not remove existing NVIDIA kernel modules not part of an earlier NVIDIA driver installation.  Please ensure that NVIDIA kernel modules matching this driver version are installed separately.

ERROR: Unable to create '/lib/firmware/nvidia/515.65.01/gsp.bin' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-smi' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-debugdump' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libcuda.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-cuda-mps-control' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-cuda-mps-server' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/bin/nvidia-persistenced' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-compiler.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-tls.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvoptix.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-ngx.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvcuvid.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-encode.so.515.65.01' for copying (Read-only file system)

ERROR: Unable to create '/usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.515.65.01' for copying (Read-only file system)

ERROR: Failed to execute `/usr/bin/systemctl daemon-reload`: System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

WARNING: The installed file '/lib/firmware/nvidia/515.65.01/gsp.bin' has permissions 0644, but it was installed with permissions 0444.

WARNING: Unable to locate/open X configuration file.

Package xorg-server was not found in the pkg-config search path.
Perhaps you should add the directory containing `xorg-server.pc'
to the PKG_CONFIG_PATH environment variable
No package 'xorg-server' found
Option "ProbeAllGpus" "False" added to Screen "Screen0".
Option "BaseMosaic" "False" added to Screen "Screen0".
Option "AllowEmptyInitialConfiguration" "True" added to Screen "Screen0".
New X configuration file written to '/etc/X11/xorg.conf'

Waiting for X socket
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

X.Org X Server 1.21.1.3
X Protocol Version 11, Revision 0
Current Operating System: Linux d9cca5990280 4.18.0-372.19.1.el8_6.x86_64 #1 SMP Mon Jul 18 11:14:02 EDT 2022 x86_64
Kernel command line: BOOT_IMAGE=(hd0,gpt2)/vmlinuz-4.18.0-372.19.1.el8_6.x86_64 root=/dev/mapper/rhel-root ro resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet fips=1 boot=UUID=64eeaa30-2659-4a54-98ad-02c72b68519d audit=1 audit_backlog_limit=8192 pti=on vsyscall=none page_poison=1 slub_debug=P rd.driver.blacklist=nouveau
xorg-server 2:21.1.3-2ubuntu2.5 (For technical support please see http://www.ubuntu.com/support) 
Current version of pixman: 0.40.0
        Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/home/user/.local/share/xorg/Xorg.0.log", Time: Mon Jan 30 20:54:50 2023
(==) Using config file: "/etc/X11/xorg.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
X socket is ready
Session Running. Press [Return] to exit.
QIODevice::read (QFile, "/home/user/.config/kdedefaults/package"): device not open
QDBusConnection: error: could not send signal to service "" path "//home/user/.config/kdedefaults/kdeglobals" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.config/kdedefaults/kdeglobals
QDBusConnection: error: could not send signal to service "" path "//home/user/.config/kdedefaults/kdeglobals" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.config/kdedefaults/kdeglobals
QDBusConnection: error: could not send signal to service "" path "//home/user/.config/kdedefaults/kdeglobals" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.config/kdedefaults/kdeglobals
QDBusConnection: error: could not send signal to service "" path "//home/user/.config/kdedefaults/kcminputrc" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.config/kdedefaults/kcminputrc
QDBusConnection: error: could not send signal to service "" path "//home/user/.config/kdedefaults/kwinrc" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.config/kdedefaults/kwinrc
QPixmap: QGuiApplication must be created before calling defaultDepth().
QPixmap: QGuiApplication must be created before calling defaultDepth().
QDBusConnection: error: could not send signal to service "" path "//home/user/.kde/share/config/kdeglobals" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.kde/share/config/kdeglobals
kdeinit5: preparing to launch '/usr/lib/x86_64-linux-gnu/libexec/kf5/klauncher'
kdeinit5: Launched KLauncher, pid = 311, result = 0
Connecting to deprecated signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)
kdeinit5: opened connection to :0
Initializing  "/usr/lib/x86_64-linux-gnu/qt5/plugins/plasma/kcms/systemsettings/kcm_fonts.so"
Initializing  "/usr/lib/x86_64-linux-gnu/qt5/plugins/plasma/kcms/systemsettings/kcm_mouse.so"
kdeinit5: Got SETENV 'XCURSOR_THEME=breeze_cursors' from launcher.
kdeinit5: Got SETENV 'XCURSOR_SIZE=24' from launcher.
Initializing  "/usr/lib/x86_64-linux-gnu/qt5/plugins/plasma/kcms/systemsettings/kcm_style.so"
kdeinit5: Got SETENV 'GTK_RC_FILES=/etc/gtk/gtkrc:/home/user/.gtkrc:/home/user/.config/gtkrc' from launcher.
kdeinit5: Got SETENV 'GTK2_RC_FILES=/etc/gtk-2.0/gtkrc:/home/user/.gtkrc-2.0:/home/user/.config/gtkrc-2.0' from launcher.
QDBusConnection: error: could not send signal to service "" path "//home/user/.kde/share/config/kdeglobals" interface "org.kde.kconfig.notify" member "ConfigChanged": Invalid object path: //home/user/.kde/share/config/kdeglobals
org.kde.plasma.session: process job  "kcminit_startup" finished with exit code  0
kcm_touchpad: Using X11 backend
Installing the delayed initialization callback.
System timezone has been changed, new timezone is "UTC"
kf.coreaddons: "Could not load plugin from colord: The shared library was not found."
kf.kded: found kded module "colord" by prepending 'kded_' to the library path, please fix your metadata.
Using XRANDR extension 1.3 or greater.
QDBusConnection: name 'org.kde.kglobalaccel' had owner '' but we thought it was ':1.7'
QDBusConnection: name 'org.freedesktop.UDisks2' had owner '' but we thought it was ':1.20'
Delayed initialization.
Reloading the khotkeys configuration
Version 2 File!
Loaded defaults from "/usr/share/khotkeys/defaults.khotkeys"
true
Imported file "/usr/share/khotkeys/defaults.khotkeys"
Version 2 File!
Imported file "/usr/share/khotkeys/kde32b1.khotkeys"
Version 2 File!
Imported file "/usr/share/khotkeys/konqueror_gestures_kde321.khotkeys"
Version 2 File!
Imported file "/usr/share/khotkeys/konsole.khotkeys"
Icc path "/home/user/.local/share/icc" does not exist
"/home/user/.local/share/icc/." is not an ICC profile
"/home/user/.local/share/icc/.." is not an ICC profile
PNP ID "NVD" Vendor Name "Nvidia"
Adding device id "xrandr-Nvidia-NVIDIA VGX"
Output Hash "f3a2059f77dd2a14b929c51b82f3ed59"
Output isLaptop false
Failed to register device: "failed to obtain org.freedesktop.color-manager.create-device auth"
"CMF_product" "colord-kde"
"CMF_binary" "colord-kde"
"CMF_version" "0.5.0"
"DATA_source" "edid"
"EDID_md5" "f3a2059f77dd2a14b929c51b82f3ed59"
"EDID_model" "NVIDIA VGX"
"EDID_mnft" "NVD"
"EDID_manufacturer" "Nvidia"
"" ":1.24"
"/home/user/.local/share/icc/." is not an ICC profile
"/home/user/.local/share/icc/.." is not an ICC profile
created profile "icc-1872cec71b302ee8d0b8b133bd583a0d" ""
PNP ID "NVD" Vendor Name "Nvidia"
Adding device id "xrandr-Nvidia-NVIDIA VGX"
Output Hash "f3a2059f77dd2a14b929c51b82f3ed59"
Output isLaptop false
Failed to register device: "failed to obtain org.freedesktop.color-manager.create-device auth"
created profile "icc-1872cec71b302ee8d0b8b133bd583a0d" ""
EDID ICC Profile already exists "/home/user/.local/share/icc/edid-f3a2059f77dd2a14b929c51b82f3ed59.icc"
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
org.kde.kmix: no mix devices and not dynamic
(python3:246): libnice-WARNING **: 20:59:03.941: Agent 0x7fb0f01593f0 : We still have alive TURN refreshes. Consider using nice_agent_close_async() to prune them before releasing the agent.
INFO:gstwebrtc_app:webrtcbin set to state NULL
INFO:gstwebrtc_app:pipeline stopped
INFO:signaling:Connection to peer ('127.0.0.1', 56082) closed, exiting handler
INFO:signaling:Disconnected from peer '1' at ('10.89.1.3', 50210)
INFO:signaling:Connected to ('127.0.0.1', 60532)
INFO:signaling:Registered peer '0' at ('127.0.0.1', 60532)
INFO:signaling:'0' command 'SESSION 1'
INFO:signaling:'0' command 'SESSION 1'
INFO:signaling:'0' command 'SESSION 1'
INFO:signaling:'0' command 'SESSION 1'
INFO:signaling:Connected to ('10.89.1.3', 38986)
INFO:signaling:'0' command 'SESSION 1'
INFO:signaling:Registered peer '1' at ('10.89.1.3', 38986)
INFO:signaling:'0' command 'SESSION 1'
INFO:signaling:Session from '0' (('127.0.0.1', 60532)) to '1' (('10.89.1.3', 38986))
INFO:gstwebrtc_app:starting pipeline
INFO:gstwebrtc_app:adding TURN server: turn://user:mypasswd@192.168.98.71:3478
INFO:gstwebrtc_app:pipeline started
INFO:gstwebrtc_app:handling on-negotiation-needed, creating offer.
0:04:24.850187061   246 0x7fb090005640 WARN                GST_PADS gstpad.c:4351:gst_pad_peer_query:<nicesrc6:src> could not send sticky events
WARNING:gstwebrtc_app:injecting rtx-time to SDP
WARNING:gstwebrtc_app:injecting profile-level-id to SDP
WARNING:gstwebrtc_app:injecting level-asymmetry-allowed to SDP
INFO:signaling:0 -> 1: {"sdp": {"type": "offer", "sdp": "v=0\r\no=- 8644791011959618801 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=ice-options:trickle\r\na=group:BUNDLE video0 audio1 application2\r\nm=video 9 UDP/TLS/RTP/SAVPF 123 97\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:NXP8kb3W9kIFcw9/EFTvQaY8NXi+2ihX\r\na=ice-pwd:PrNS8ECzvrh33l0bP24rBxZ8GPj83ieb\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=sendrecv\r\na=rtpmap:123 H264/90000\r\na=rtcp-fb:123 nack\r\na=rtcp-fb:123 nack pli\r\na=rtcp-fb:123 ccm fir\r\na=rtcp-fb:123 transport-cc\r\na=framerate:100\r\na=fmtp:123 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1;sprop-parameter-sets=Z2QAM6wrQDwBE/LgLUBAQFAAAD6AADDUCEA=,aO48sA==;aggregate-mode=zero-latency\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=123;rtx-time=125\r\na=ssrc-group:FID 945692935 1439742178\r\na=ssrc:945692935 msid:user3924485558@host-6addbfd1 webrtctransceiver12\r\na=ssrc:945692935 cname:user3924485558@host-6addbfd1\r\na=ssrc:1439742178 msid:user3924485558@host-6addbfd1 webrtctransceiver12\r\na=ssrc:1439742178 cname:user3924485558@host-6addbfd1\r\na=mid:video0\r\na=fingerprint:sha-256 02:FC:6A:31:3B:76:FD:38:33:D2:EC:A9:62:F2:DD:92:47:9A:92:66:42:C3:29:23:3E:AB:1B:1D:02:BE:47:C0\r\na=rtcp-mux-only\r\nm=audio 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:NXP8kb3W9kIFcw9/EFTvQaY8NXi+2ihX\r\na=ice-pwd:PrNS8ECzvrh33l0bP24rBxZ8GPj83ieb\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=sendrecv\r\na=rtpmap:96 OPUS/48000/2\r\na=rtcp-fb:96 transport-cc\r\na=fmtp:96 sprop-stereo=1;sprop-maxcapturerate=48000\r\na=ssrc:1522969934 msid:user3924485558@host-6addbfd1 webrtctransceiver13\r\na=ssrc:1522969934 cname:user3924485558@host-6addbfd1\r\na=mid:audio1\r\na=fingerprint:sha-256 02:FC:6A:31:3B:76:FD:38:33:D2:EC:A9:62:F2:DD:92:47:9A:92:66:42:C3:29:23:3E:AB:1B:1D:02:BE:47:C0\r\na=rtcp-mux-only\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=setup:actpass\r\na=ice-ufrag:NXP8kb3W9kIFcw9/EFTvQaY8NXi+2ihX\r\na=ice-pwd:PrNS8ECzvrh33l0bP24rBxZ8GPj83ieb\r\na=bundle-only\r\na=mid:application2\r\na=sctp-port:5000\r\na=fingerprint:sha-256 02:FC:6A:31:3B:76:FD:38:33:D2:EC:A9:62:F2:DD:92:47:9A:92:66:42:C3:29:23:3E:AB:1B:1D:02:BE:47:C0\r\n"}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:1 1 UDP 2015363327 10.89.1.3 40139 typ host", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:2 1 TCP 1015021823 10.89.1.3 9 typ host tcptype active", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:3 1 TCP 1010827519 10.89.1.3 52235 typ host tcptype passive", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:4 1 UDP 2015363583 fe80::70b7:acff:fe0a:8a22 52474 typ host", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:5 1 TCP 1015022079 fe80::70b7:acff:fe0a:8a22 9 typ host tcptype active", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:6 1 TCP 1010827775 fe80::70b7:acff:fe0a:8a22 43879 typ host tcptype passive", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:7 1 TCP 847249663 192.168.98.71 9 typ srflx raddr 10.89.1.3 rport 9 tcptype active", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:8 1 TCP 843055359 192.168.98.71 52235 typ srflx raddr 10.89.1.3 rport 52235 tcptype passive", "sdpMLineIndex": 0}}
INFO:signaling:0 -> 1: {"ice": {"candidate": "candidate:9 1 UDP 337658111 192.168.98.71 49462 typ relay raddr 10.89.1.3 rport 9", "sdpMLineIndex": 0}}
INFO:signaling:1 -> 0: {"sdp":{"type":"answer","sdp":"v=0\r\no=- 2098574401704134513 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE video0 audio1 application2\r\na=msid-semantic: WMS\r\nm=video 9 UDP/TLS/RTP/SAVPF 123 97\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:xgti\r\na=ice-pwd:LrrUF5mEoPel3ls4XPkOE/tS\r\na=ice-options:trickle\r\na=fingerprint:sha-256 F6:A8:F3:2B:E9:80:F2:56:83:36:CA:3E:FB:B3:6A:4D:36:04:CC:4D:94:22:BE:CA:2C:49:F8:43:67:88:8C:25\r\na=setup:active\r\na=mid:video0\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:123 H264/90000\r\na=rtcp-fb:123 transport-cc\r\na=rtcp-fb:123 ccm fir\r\na=rtcp-fb:123 nack\r\na=rtcp-fb:123 nack pli\r\na=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=123;rtx-time=125\r\nm=audio 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:xgti\r\na=ice-pwd:LrrUF5mEoPel3ls4XPkOE/tS\r\na=ice-options:trickle\r\na=fingerprint:sha-256 F6:A8:F3:2B:E9:80:F2:56:83:36:CA:3E:FB:B3:6A:4D:36:04:CC:4D:94:22:BE:CA:2C:49:F8:43:67:88:8C:25\r\na=setup:active\r\na=mid:audio1\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:96 OPUS/48000/2\r\na=rtcp-fb:96 transport-cc\r\na=fmtp:96 minptime=10;stereo=1;useinbandfec=1\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:xgti\r\na=ice-pwd:LrrUF5mEoPel3ls4XPkOE/tS\r\na=ice-options:trickle\r\na=fingerprint:sha-256 F6:A8:F3:2B:E9:80:F2:56:83:36:CA:3E:FB:B3:6A:4D:36:04:CC:4D:94:22:BE:CA:2C:49:F8:43:67:88:8C:25\r\na=setup:active\r\na=mid:application2\r\na=sctp-port:5000\r\n"}}
INFO:signaling:1 -> 0: {"ice":{"candidate":"candidate:4672349 1 udp 16785407 192.168.98.71 49440 typ relay raddr 0.0.0.0 rport 0 generation 0 ufrag xgti network-cost 999","sdpMid":"video0","sdpMLineIndex":0}}
INFO:gstwebrtc_app:setting ICE candidate: 0, candidate:4672349 1 udp 16785407 192.168.98.71 49440 typ relay raddr 0.0.0.0 rport 0 generation 0 ufrag xgti network-cost 999
0:04:27.353724120   246 0x7fb0f0001b60 WARN                 srtpenc gstsrtpenc.c:1090:gst_srtp_enc_check_set_caps:<srtpenc6> error: Could not initialize SRTP encoder
0:04:27.353827601   246 0x7fb0f0001b60 WARN                 srtpenc gstsrtpenc.c:1090:gst_srtp_enc_check_set_caps:<srtpenc6> error: Failed to add stream to SRTP encoder (err: 5)
0:04:27.354059729   246 0x7fb0f00f8800 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<x11> error: Internal data stream error.
0:04:27.354080019   246 0x7fb0f00f8800 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<x11> error: streaming stopped, reason error (-5)
0:04:27.354775938   246 0x7fb0f00f8800 WARN                   queue gstqueue.c:992:gst_queue_handle_sink_event:<queue13> error: Internal data stream error.
0:04:27.354794774   246 0x7fb0f00f8800 WARN                   queue gstqueue.c:992:gst_queue_handle_sink_event:<queue13> error: streaming stopped, reason error (-5)
0:04:27.368102443   246 0x7fb098005760 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<pulsesrc> error: Internal data stream error.
0:04:27.368126541   246 0x7fb098005760 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<pulsesrc> error: streaming stopped, reason error (-5)
0:04:27.368309143   246 0x7fb098005760 WARN                   queue gstqueue.c:992:gst_queue_handle_sink_event:<rtpopuspay_queue> error: Internal data stream error.
0:04:27.368316212   246 0x7fb098005760 WARN                   queue gstqueue.c:992:gst_queue_handle_sink_event:<rtpopuspay_queue> error: streaming stopped, reason error (-5)
0:04:27.407903514   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:27.407925907   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:27.407932346   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
INFO:main:opened peer data channel for user input to X11
INFO:gstwebrtc_app:sending frame rate
INFO:gstwebrtc_app:sending video bitrate
INFO:gstwebrtc_app:sending audio bitrate
INFO:gstwebrtc_app:sending audio enabled
INFO:gstwebrtc_app:sending resize enabled state
INFO:gstwebrtc_app:sending encoder: nvh264enc
INFO:webrtc_input:Setting framerate to: 100
INFO:gstwebrtc_app:framerate set to: 100
INFO:webrtc_input:Setting video bitrate to: 4000
INFO:gstwebrtc_app:video bitrate set to: 4000
INFO:webrtc_input:Setting audio bitrate to: 128000
INFO:gstwebrtc_app:audio bitrate set to: 128000
INFO:webrtc_input:Setting enable_audio to: True
INFO:gstwebrtc_app:audio set to: True
INFO:webrtc_input:Setting enable_resize to : False
INFO:main:removing handler for on_resize
INFO:webrtc_input:Resetting keyboard modifiers.
0:04:29.560110302   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:29.560381307   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 1
0:04:29.560395149   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
0:04:32.312885505   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:32.312919779   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:32.312928178   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
0:04:35.000477454   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:35.000505781   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:35.000512591   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
0:04:40.720252264   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:40.720282335   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:40.720289418   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
INFO:webrtc_input:Resetting keyboard modifiers.
0:04:46.937910273   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:46.937929319   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:46.937934988   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
0:04:51.861859322   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:51.861905224   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:51.861913242   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
0:04:56.094953831   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:04:56.094980970   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:04:56.094988667   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
0:05:03.187197712   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:939:update_session_stream_from_caps:<srtpdec6> Failed to create the stream (err: 5)
0:05:03.187223746   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:995:request_key_with_signal:<srtpdec6> Could not set stream with SSRC 4195875351
0:05:03.187230580   246 0x7fb090005e40 WARN                 srtpdec gstsrtpdec.c:1466:gst_srtp_dec_chain:<srtpdec6> Invalid buffer, dropping
mixxen commented 1 year ago

Just FYI. I rebuilt all the containers and reran on Chrome 111.0.563.65, and everything seems to be working again.

ehfd commented 1 year ago

I'm closing this because other than stats issues, they work.

ehfd commented 8 months ago

Happens now.

I believe that this issue is related to Windows 11 Efficiency Mode (green leaf shown in Task Manager).

Best effort to try for now: Start > Settings > System > Power & battery > Power mode > Best performance

ehfd commented 8 months ago

Moreover:

powerEfficientDecoder: true in webrtc-internals must be set to false somehow.

https://github.com/w3c/webrtc-stats/issues/730 https://github.com/w3c/webrtc-stats/issues/666 https://www.w3.org/TR/webrtc-stats/#dom-rtcinboundrtpstreamstats-powerefficientdecoder

ehfd commented 8 months ago

image

ehfd commented 8 months ago

I added keyframes in https://github.com/selkies-project/selkies-gstreamer/commit/335d2597ff341b91c429bef666887c4ba957d6ae to mediate stream resilience, but this seems like a totally different issue and happens at the start.

Two things happen: 1. FFmpeg (fallback from: ExternalDecoder (D3D11VideoDecoder)) 2. One keyframe is decoded, streams keep coming in, then everything drops.

ssrc | 3177285588
-- | --
kind | video
transportId | Tvideo01
codecId | CITvideo01_123_level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
[codec] | H264 (123, level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f)
jitter | 0.001
packetsLost | 960
trackIdentifier | webrtctransceiver7
mid | video0
packetsReceived | 385759
[packetsReceived/s] | 943.770850884583
bytesReceived | 518210865
[bytesReceived_in_bits/s] | 10108086.0252738
headerBytesReceived | 4629108
[headerBytesReceived_in_bits/s] | 90602.00168491996
retransmittedPacketsReceived | 952
[retransmittedPacketsReceived/s] | 55.21145745577085
retransmittedBytesReceived | 1286011
[retransmittedBytesReceived_in_bits/s] | 600272.7683235046
rtxSsrc | 4032281640
lastPacketReceivedTimestamp | 1712079536907.719
[lastPacketReceivedTimestamp] | 2024. 4. 3. 오전 2:38:56
jitterBufferDelay | 17.253201
[jitterBufferDelay/jitterBufferEmittedCount_in_ms] | 0
jitterBufferTargetDelay | 5.224175
[jitterBufferTargetDelay/jitterBufferEmittedCount_in_ms] | 0
jitterBufferMinimumDelay | 5.224175
[jitterBufferMinimumDelay/jitterBufferEmittedCount_in_ms] | 0
jitterBufferEmittedCount | 1162
framesReceived | 120
[framesReceived/s] | 0
[framesReceived-framesDecoded-framesDropped] | -1042
frameWidth | 1920
frameHeight | 1080
framesDecoded | 120
[framesDecoded/s] | 0
keyFramesDecoded | 1
[keyFramesDecoded/s] | 0
framesDropped | 1042
totalDecodeTime | 2.341894
[totalDecodeTime/framesDecoded_in_ms] | 0
totalProcessingDelay | 2.494303
[totalProcessingDelay/framesDecoded_in_ms] | 0
totalAssemblyTime | 0.024973
[totalAssemblyTime/framesAssembledFromMultiplePackets_in_ms] | 0
framesAssembledFromMultiplePackets | 101
totalInterFrameDelay | 1.829
[totalInterFrameDelay/framesDecoded_in_ms] | 0
totalSquaredInterFrameDelay | 0.07186100000000002
[interFrameDelayStDev_in_ms] | 0
pauseCount | 0
totalPausesDuration | 0
freezeCount | 0
totalFreezesDuration | 0
decoderImplementation | FFmpeg (fallback from: ExternalDecoder (D3D11VideoDecoder))
firCount | 0
pliCount | 1810
nackCount | 399
powerEfficientDecoder | false
minPlayoutDelay | 0
ehfd commented 8 months ago

https://github.com/selkies-project/selkies-gstreamer/commit/dd9bef9841a8b8a494423de9e587130f066685c2

Fixed a lot of things. Please help test.

ehfd commented 8 months ago

Congestion Control: You can use the rtpgccbwe element which will signal the estimated bandwidth (with notify::estimate-bandwidth) and set the encoder bitrate property from there to use GCC Congestion Control.

vamman commented 8 months ago

I added keyframes in 335d259 to mediate stream resilience, but this seems like a totally different issue and happens at the start.

Two things happen: 1. FFmpeg (fallback from: ExternalDecoder (D3D11VideoDecoder)) 2. One keyframe is decoded, streams keep coming in, then everything drops.

ssrc | 3177285588
-- | --
kind | video
transportId | Tvideo01
codecId | CITvideo01_123_level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
[codec] | H264 (123, level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f)
jitter | 0.001
packetsLost | 960
trackIdentifier | webrtctransceiver7
mid | video0
packetsReceived | 385759
[packetsReceived/s] | 943.770850884583
bytesReceived | 518210865
[bytesReceived_in_bits/s] | 10108086.0252738
headerBytesReceived | 4629108
[headerBytesReceived_in_bits/s] | 90602.00168491996
retransmittedPacketsReceived | 952
[retransmittedPacketsReceived/s] | 55.21145745577085
retransmittedBytesReceived | 1286011
[retransmittedBytesReceived_in_bits/s] | 600272.7683235046
rtxSsrc | 4032281640
lastPacketReceivedTimestamp | 1712079536907.719
[lastPacketReceivedTimestamp] | 2024. 4. 3. 오전 2:38:56
jitterBufferDelay | 17.253201
[jitterBufferDelay/jitterBufferEmittedCount_in_ms] | 0
jitterBufferTargetDelay | 5.224175
[jitterBufferTargetDelay/jitterBufferEmittedCount_in_ms] | 0
jitterBufferMinimumDelay | 5.224175
[jitterBufferMinimumDelay/jitterBufferEmittedCount_in_ms] | 0
jitterBufferEmittedCount | 1162
framesReceived | 120
[framesReceived/s] | 0
[framesReceived-framesDecoded-framesDropped] | -1042
frameWidth | 1920
frameHeight | 1080
framesDecoded | 120
[framesDecoded/s] | 0
keyFramesDecoded | 1
[keyFramesDecoded/s] | 0
framesDropped | 1042
totalDecodeTime | 2.341894
[totalDecodeTime/framesDecoded_in_ms] | 0
totalProcessingDelay | 2.494303
[totalProcessingDelay/framesDecoded_in_ms] | 0
totalAssemblyTime | 0.024973
[totalAssemblyTime/framesAssembledFromMultiplePackets_in_ms] | 0
framesAssembledFromMultiplePackets | 101
totalInterFrameDelay | 1.829
[totalInterFrameDelay/framesDecoded_in_ms] | 0
totalSquaredInterFrameDelay | 0.07186100000000002
[interFrameDelayStDev_in_ms] | 0
pauseCount | 0
totalPausesDuration | 0
freezeCount | 0
totalFreezesDuration | 0
decoderImplementation | FFmpeg (fallback from: ExternalDecoder (D3D11VideoDecoder))
firCount | 0
pliCount | 1810
nackCount | 399
powerEfficientDecoder | false
minPlayoutDelay | 0

As I reported on Discord we had this issue for a few weeks and seriously struggled with it. It seemed to come out of nowhere and it seemed like any new version of Chromium was having the issue. However, upon deeper investigation it became very clear the issue was actually being aggravated by traffic shaping with one of our providers. We had the exact symptoms of one frame arriving. I'm very interested in your work to solve this using GCC. If you turn off hardware acceleration I bet it will work like it did before. But this is NOT a solution for average users.

ehfd commented 7 months ago

https://github.com/selkies-project/selkies-gstreamer/commit/dd9bef9841a8b8a494423de9e587130f066685c2

@vamman this commit fixed the core culprit of the issue I saw. Build assets are available in GitHub Actions. And the issue was also when hardware acceleration was OFF as well as ON.

I feel that congestion control is a bonus at this stage, but I'll look at it.

ehfd commented 7 months ago

Also need to experiment with FEC settings with video for stream redundancy (currently implemented in audio).

ehfd commented 7 months ago

FEC isn't as effective as I've expected; rtpgccbwe is probably the thing left I can do.

ehfd commented 7 months ago

Stability improvement is immense with the latest commit, assuming that the packet drop is not ridiculous.

Please test and feedback from main.