Kurento / bugtracker

[ARCHIVED] Contents migrated to monorepo: https://github.com/Kurento/kurento
46 stars 10 forks source link

Kurento eats 100% of CPU and becomes unresponsive in RTP receiver scenario #591

Open makkarpov opened 3 years ago

makkarpov commented 3 years ago

Prerequisites

Issue description

We have simple RTP-to-WebRTC broadcast scenario with One-to-Many Dispatcher in the middle, like this:

image

Now, if I go to the browser and hit F5 couple of times (each time letting Kurento fully create new port and WebRTC endpoint and complete WebRTC negotiation, so picture appears), it becomes unresponsive very soon. I encountered this on 20th refresh. When this happens, Kurento eats all available CPU cores. It can recover automatically in tenth of seconds, or it can stay in this state for a tenth of minutes. The general trend is that more aggressive refreshes lead to longer outage times.

image

Both RTP source and negotiated WebRTC codecs are H.264, so Kurento does not transcode here, it just acting as a packet forwarder.

How to reproduce?

Create pipeline as on picture and add WebRTC receivers to it.

Expected & current behavior

Does not hang & hangs.

About Kurento Media Server

About your Application Server

About end-user clients

Run these commands

Ubuntu 16.04.7 LTS

Kurento Media Server version: 6.16.0
Found modules:
    'chroma' version 6.16.0
    'core' version 6.16.0
    'crowddetector' version 6.16.0
    'elements' version 6.16.0
    'filters' version 6.16.0
    'platedetector' version 6.16.0
    'pointerdetector' version 6.16.0
DPKG package list ``` ii gstreamer1.5-alsa:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer plugin for ALSA ii gstreamer1.5-libav:amd64 1.8.1-1kurento1.16.04 amd64 libav plugin for GStreamer ii gstreamer1.5-libav-dbg:amd64 1.8.1-1kurento1.16.04 amd64 libav plugin for GStreamer (debug symbols) ii gstreamer1.5-nice:amd64 0.1.18-0kurento1.16.04 amd64 ICE library (GStreamer 1.5 plugin) ii gstreamer1.5-nice-dbgsym:amd64 0.1.18-0kurento1.16.04 amd64 debug symbols for package gstreamer1.5-nice ii gstreamer1.5-plugins-bad:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer plugins from the "bad" set ii gstreamer1.5-plugins-bad-dbg:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer plugins from the "bad" set (debug symbols) ii gstreamer1.5-plugins-base:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer plugins from the "base" set ii gstreamer1.5-plugins-base-dbg:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer plugins from the "base" set ii gstreamer1.5-plugins-good:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer plugins from the "good" set ii gstreamer1.5-plugins-good-dbg:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer plugins from the "good" set ii gstreamer1.5-plugins-ugly:amd64 1.8.1-1kurento1.16.04 amd64 GStreamer plugins from the "ugly" set ii gstreamer1.5-plugins-ugly-dbg:amd64 1.8.1-1kurento1.16.04 amd64 GStreamer plugins from the "ugly" set (debug symbols) ii gstreamer1.5-pulseaudio:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer plugin for PulseAudio ii gstreamer1.5-x:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer plugins for X11 and Pango ii kms-chroma 6.16.0-0kurento1.16.04 amd64 Kurento Chroma filter ii kms-core 6.16.0-0kurento1.16.04 amd64 Kurento Core module ii kms-core-dbg 6.16.0-0kurento1.16.04 amd64 Kurento Core module ii kms-crowddetector 6.16.0-0kurento1.16.04 amd64 Kurento Crowd Detector filter ii kms-elements 6.16.0-0kurento1.16.04 amd64 Kurento Elements module ii kms-elements-dbg 6.16.0-0kurento1.16.04 amd64 Kurento Elements module ii kms-filters 6.16.0-0kurento1.16.04 amd64 Kurento Filters module ii kms-filters-dbg 6.16.0-0kurento1.16.04 amd64 Kurento Filters module ii kms-jsonrpc 6.16.0-0kurento1.16.04 amd64 Kurento JSON-RPC library ii kms-jsonrpc-dbg 6.16.0-0kurento1.16.04 amd64 Kurento JSON-RPC library ii kms-platedetector 6.16.0-0kurento1.16.04 amd64 Kurento Plate Detector filter ii kms-pointerdetector 6.16.0-0kurento1.16.04 amd64 Kurento Pointer Detector filter ii kmsjsoncpp 1.6.3-1kurento1.16.04 amd64 Kurento jsoncpp library ii kmsjsoncpp-dbg 1.6.3-1kurento1.16.04 amd64 Kurento jsoncpp library ii kurento-dbg 6.16.0-0kurento1.16.04 amd64 Meta-package that installs debug symbols ii kurento-media-server 6.16.0-0kurento1.16.04 amd64 Kurento Media Server ii kurento-media-server-dbg 6.16.0-0kurento1.16.04 amd64 Kurento Media Server ii libgstreamer-plugins-bad1.5-0:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer development files for libraries from the "bad" set ii libgstreamer-plugins-base1.5-0:amd64 1.8.1-1kurento2.16.04 amd64 GStreamer libraries from the "base" set ii libgstreamer1.5-0:amd64 1.8.1-1kurento2.16.04 amd64 Core GStreamer libraries and elements ii libgstreamer1.5-0-dbg:amd64 1.8.1-1kurento2.16.04 amd64 Core GStreamer libraries and elements ii libnice10:amd64 0.1.18-0kurento1.16.04 amd64 ICE library (shared library) ii libnice10-dbgsym:amd64 0.1.18-0kurento1.16.04 amd64 debug symbols for package libnice10 ii libsrtp0:amd64 1.6.0-0kurento1.16.04 amd64 Secure RTP (SRTP) and UST Reference Implementations - shared library ii libusrsctp 0.9.2-1kurento1.16.04 amd64 sctp-refimpl library ii openh264 1.5.0-0kurento1.16.04 amd64 H.264 Video Codec provided by Cisco Systems, Inc. ii openh264-gst-plugins-bad-1.5:amd64 1.8.1-1kurento5.16.04 amd64 GStreamer plugins from openh264 ii openwebrtc-gst-plugins 0.10.0-1kurento1.16.04 amd64 OpenWebRTC specific GStreamer plugins ii openwebrtc-gst-plugins-dbg 0.10.0-1kurento1.16.04 amd64 OpenWebRTC specific GStreamer plugins ```
github-actions[bot] commented 3 years ago

Hello @makkarpov! :wave: we're sorry you found a bug... so first of all, thank you very much for reporting it.

To know about progress, check in Triage. All issues are considered Backlog Candidates until work priorities align and the issue is selected for development. It will then become part of our official Backlog.