KMS 6.11.0 Composite video is be blocked when participant join or leave room.

[sshsun1990]

KMS Version:

Kurento Media Server version: 6.11.0 Found modules: 'core' version 6.11.0 'elements' version 6.11.0 'filters' version 6.11.0

Ubuntu Version

No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.6 LTS Release: 16.04 Codename: xenial

Other libraries versions:

ii gir1.2-gst-plugins-base-1.5 1.8.1-1kurento1.16.04 amd64 Description: GObject introspection data for the GStreamer Plugins Base library ii gir1.2-gstreamer-1.5 1.8.1-1kurento1.16.04 amd64 Description: GObject introspection data for the GStreamer library ii gir1.2-nice-0.1:amd64 0.1.15-1kurento3.16.04 amd64 ICE library (GObject introspection) ii gstreamer1.5-libav:amd64 1.8.1-1kurento1.16.04 amd64 libav plugin for GStreamer ii gstreamer1.5-nice:amd64 0.1.15-1kurento3.16.04 amd64 ICE library (GStreamer 1.5 plugin) ii gstreamer1.5-plugins-bad:amd64 1.8.1-1kurento3.16.04 amd64 GStreamer plugins from the "bad" set ii gstreamer1.5-plugins-base:amd64 1.8.1-1kurento1.16.04 amd64 GStreamer plugins from the "base" set ii gstreamer1.5-plugins-good:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer plugins from the "good" set ii gstreamer1.5-plugins-ugly:amd64 1.8.1-1kurento1.16.04 amd64 GStreamer plugins from the "ugly" set ii gstreamer1.5-pulseaudio:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer plugin for PulseAudio ii gstreamer1.5-x:amd64 1.8.1-1kurento1.16.04 amd64 GStreamer plugins for X11 and Pango ii kms-cmake-utils 6.11.0-0kurento1.16.04 all Kurento CMake utils ii kms-cmake-utils-build-deps 6.11.1-0kurento1 all build-dependencies for kms-cmake-utils ii kms-core 6.11.0-0kurento1.16.04 amd64 Kurento Core module ii kms-core-build-deps 6.11.1-0kurento1 all build-dependencies for kms-core ii kms-core-dev 6.11.0-0kurento1.16.04 amd64 Kurento Core module ii kms-elements 6.11.0-0kurento1.16.04 amd64 Kurento Elements module ii kms-elements-build-deps 6.11.1-0kurento1 all build-dependencies for kms-elements ii kms-elements-dev 6.11.0-0kurento1.16.04 amd64 Kurento Elements module ii kms-filters 6.11.0-0kurento1.16.04 amd64 Kurento Filters module ii kms-filters-build-deps 6.11.1-0kurento1 all build-dependencies for kms-filters ii kms-filters-dev 6.11.0-0kurento1.16.04 amd64 Kurento Filters module ii kms-jsonrpc 6.11.0-0kurento1.16.04 amd64 Kurento JSON-RPC library ii kms-jsonrpc-build-deps 6.11.1-0kurento1 all build-dependencies for kms-jsonrpc ii kms-jsonrpc-dev 6.11.0-0kurento1.16.04 amd64 Kurento JSON-RPC library ii kmsjsoncpp 1.6.3-1kurento1.16.04 amd64 Kurento jsoncpp library ii kmsjsoncpp-dev 1.6.3-1kurento1.16.04 amd64 Kurento jsonrpc library ii kurento-media-server 6.11.0-0kurento1.16.04 amd64 Kurento Media Server ii kurento-media-server-build-deps 6.11.1-0kurento1 all build-dependencies for kurento-media-server ii kurento-module-creator 6.11.0-0kurento1.16.04 all Kurento Module Creator ii kurento-module-creator-build-deps 6.11.1-0kurento1 all build-dependencies for kurento-module-creator ii libgstreamer-plugins-bad1.5-0:amd64 1.8.1-1kurento3.16.04 amd64 GStreamer development files for libraries from the "bad" set ii libgstreamer-plugins-base1.5-0:amd64 1.8.1-1kurento1.16.04 amd64 GStreamer libraries from the "base" set ii libgstreamer-plugins-base1.5-dev 1.8.1-1kurento1.16.04 amd64 GStreamer development files for libraries from the "base" set ii libgstreamer1.5-0:amd64 1.8.1-1kurento1.16.04 amd64 Core GStreamer libraries and elements ii libgstreamer1.5-dev 1.8.1-1kurento1.16.04 amd64 GStreamer core development files ii libnice-dev 0.1.15-1kurento3.16.04 amd64 ICE library (development files) ii libnice10:amd64 0.1.15-1kurento3.16.04 amd64 ICE library (shared library) ii libreoffice-avmedia-backend-gstreamer 1:5.1.6~rc2-0ubuntu1~xenial10 amd64 GStreamer backend for LibreOffice ii libusrsctp 0.9.2-1kurento1.16.04 amd64 sctp-refimpl library ii openh264 1.4.0-1kurento1.16.04 amd64 OpenH264 library ii openh264-gst-plugins-bad-1.5:amd64 1.8.1-1kurento3.16.04 amd64 GStreamer plugins from openh264 ii openwebrtc-gst-plugins 0.10.0-1kurento1.16.04 amd64 OpenWebRTC specific GStreamer plugins ii openwebrtc-gst-plugins-dev 0.10.0-1kurento1.16.04 amd64 OpenWebRTC specific GStreamer plugins

Client libraries

• Language: Java
• Version: 2.11.0

System description:

I refer offical document, build kurento from sources used 6.11.0 version. And the java client cloned from openvidu github, used 2.11.0 version. Not use STUN/TURN.

In addition, I used installed kurento media server. the location: /usr/bin/kurento-media-server, the problem still exist.

What steps will reproduce the problem?

1. Have a moderator joinRoom, used MCU Composite (has audio and video);
2. Another an participant repeated join room and leave room;

What is the expected result? Composite video and audio is normal when participant repeated join room and leave room.

What happens instead? Composite video is blocked, audio is ok.

Does it happen with one of the tutorials?

Please provide any additional information below I add probe(buffer type) src pad of compositor, it not output when video is blocked. And when the video is blocked, I found the framerate of src pad caps will be changed. The framerate begin is (fraction)15/1, it will be changed (fraction)23/1 or 30/1 when have participant join or leave room.

the pipeline dot:

[sshsun1990]

Hi, now I attempt force compositor src pad caps framerate 15fps, not used best_fps_n, the problem is not exist. But I want to know the reason.

[nightwolftamb]

Confirm, we have same problem on Ubuntu 18.04 and Kurento 6.13. Video is blocking for 1-2 seconds when user get out of composite. Also when 2 users get out almost at the same time - composite output blocking for 10-20 seconds. Very critical. The wrokaround for us was waiting for 10 seconds before disposing HubPort of one of get-outed user.

[sshsun1990]

Oh, yes, I attempt the way too some time ago, but I think it's not a good way.

So I see the source code of compositor plugins(gst-plugins-bad), I found the framerate changed when the video is blocked. Last I am try to do force caps framerate 15fps, the problem is not exists. The change is here: gst-plugins-bad/gst/compositor/compositor.c:_fixate_caps(master branch from Kurento repository)

You can try it. @nightwolftamb

[sshsun1990]

How many dealy time in your service? Perhaps, Can you tell me how trace the delay? I found the Kurento delay about 280ms ~ 370ms or bigger. Did you found them? @nightwolftamb

[nightwolftamb]

@sshsun1990 in our situation the delay in not critical. We restream composited video to youtube/fb with ffmpeg and there is delay i think near 5 seconds. If you need this data - I can check it, but how to do that? We have rather big machine with Kurento, it is 48-cores Intel Xeon.

About your fix - I did not find it on https://github.com/Kurento/gst-plugins-bad/blob/master/gst/compositor/compositor.c Is it public?

[sshsun1990]

No, it's not public.

I add a line "best_fps_n=15;" in source code: https://github.com/Kurento/gst-plugins-bad/blob/17d48dd01a53f1046cecba644fa4b7daa48d0d31/gst/compositor/compositor.c#L951

[nightwolftamb]

Ok, I will try today to put exact value there. We are using 30FPS, so will put best_fps_n = 30

[nightwolftamb]

In latest GStreamer vieomixer there are lot of code after this "if". https://github.com/GStreamer/gst-plugins-good/blob/master/gst/videomixer/videomixer2.c

[nightwolftamb]

Wrong link in last message. https://github.com/GStreamer/gst-plugins-base/blob/master/gst/compositor/compositor.c

[sshsun1990]

Maybe you should use Kurento gst-plugins-bad and other plugins. And only recompile gst-plugins-bad plugins is ok.

[nightwolftamb]

They say that kurento use forked GStreamer, so it will not work with new gstreamer out of the box. We need to wait for Kurento 7.0

[sshsun1990]

The official is working to get rid of this fork. Perhaps, near future Kurento package will use Ubuntu system libraries. Good luck to you.