RTSP/WebRTC Stream periodically freezing

NestorWetter commented 1 year ago

Which version are you using?

v0.22.0

Which operating system are you using?

[ ] Linux amd64 standard
[x] Linux amd64 Docker
[ ] Linux arm64 standard
[ ] Linux arm64 Docker
[ ] Linux arm7 standard
[ ] Linux arm7 Docker
[ ] Linux arm6 standard
[ ] Linux arm6 Docker
[ ] Windows amd64 standard
[ ] Windows amd64 Docker (WSL backend)
[ ] macOS amd64 standard
[ ] macOS amd64 Docker
[ ] Other (please describe)

Describe the issue

We're trying to stream a screen capture from a windows VM in Azure. We use ffmpeg and gdigrab to capture the desktop content and send it over rtsp to an rtsp-simple-server running on docker in another VM (also on azure). When we then try to view the stream using the built-in WebRTC player, the stream mostly works, but it periodically freezes. It usually freezes for 5-10s at least every 2 minutes. Is there perhaps a way to reduce this? Or do you know how we could analyze this further? Any help you can provide would be greatly appreciated :)

Describe how to replicate the issue

set up two VMs:
- Windows-VM:/home/wetter/Work/remote-control/RemoteControl/poc/rtsp-simple-server-poc/github-issue/diff.yml
- Azure "Standard D4s v3 (4 vcpus, 16 GiB memory)"
- Windows Server 2022 Datacenter Azure Edition
- Ubuntu-VM:
- Standard DS3 v2 (4 vcpus, 14 GiB memory)
- Ubuntu 22.04

Start RTSP Simple Server with the following docker compose config and server configuration:

docker-compose.yaml

version: '3.8'
services:
rtsp-simple-proxy:
image: aler9/rtsp-simple-server:latest
# networks:
#  - original
volumes:
- /home/azure_user/video-proxy/mount:/mount
- /home/azure_user/video-proxy/config/rtsp-simple-server.yml:/rtsp-simple-server.yml
restart: always
ports:
    - 8554:8554
    - 1935:1935
    - 8888:8888
    - 8889:8889
    - 8189:8189
    - 8322:8322
expose:
- 8554
- 8322
- 1935
- 8888
- 8889
- 8189

rtsp-simple-server.yml (these are just the differences to the default values):

# Timeout of read operations.
readTimeout: 40s
# Timeout of write operations.
writeTimeout: 40s
protocols: [tcp]
hlsEncryption: yes
#### hlsServerKey: server.key
hlsServerKey: ./mount/server.key
# Path to the server certificate.
#### hlsServerCert: server.crt
hlsServerCert: ./mount/server.crt
hlsAlwaysRemux: yes
hlsSegmentDuration: 300ms
hlsPartDuration: 100ms
paths:
all:
# If sourceOnDemand is "yes", readers will be put on hold until the source is
# ready or until this amount of time has passed.
sourceOnDemandStartTimeout: 40s
# If sourceOnDemand is "yes", the source will be closed when there are no
# readers connected and this amount of time has passed.
sourceOnDemandCloseAfter: 40s
rpiCameraDenoise: 'off'
# Readers will be put on hold until the runOnDemand command starts publishing
# or until this amount of time has passed.
runOnDemandStartTimeout: 40s
# The command will be closed when there are no
# readers connected and this amount of time has passed.
runOnDemandCloseAfter: 40s

Start FFMPEG stream with following command:
- ffmpeg -r 10 -f gdigrab -i desktop -pix_fmt yuv420p -vcodec libx264 -b:v 1024k -maxrate 2058k -bufsize 2058k -profile:v main -preset ultrafast -tune zerolatency -bf 0 -f rtsp rtsps://<SOMEUSER>:<SOMEPASSWORD>@<UBUNTU_VM_IP>:8322/<STREAM_NAME>
View Stream at WebRTC address and wait for freezing
- https://<UBUNTU_VM_IP>:8889/<STREAM_NAME>/
- Freezing seems more common when the streamed video is more dynamic (like a video with changing scenes)

Did you attach the server logs?

yes shortlogs.txt

Did you attach a network dump?

no, we did analyze the network traffic with wireshark, but found no changes in the traffic as the freezes were occurring. If its still necessary we can still find the tcpdump and provide it here.

alpe12 commented 1 year ago

I have this problem too when sourcing an RTSP from my ip camera. Opening the camera directly there's no freezes. But opening via mediamtx freezes at least once every 2 minutes. I have tried udp/tcp and changing readBufferCount to 65536. No difference.

PS: It occurs WAY more frequently over WEBRTC. And it freezes all webrtc clients at the same time. Arm64/docker.

jhcho-hanvision commented 1 year ago

I have this problem too. I have tested with some pre-recorded mp4 file, and found that it freezes the same play position. I hope this information can help you fix this bug. With this 1 and half minute mp4 file, it freezes at 54:09 and 1:21:19. Here is my console command for this test.

ffmpeg -stream_loop -1 -re -i rtmp.mp4 -an -c:v copy -f flv rtmp://127.0.0.1/spark

Here is the mp4 link. https://1drv.ms/v/s!Ah_4wJEkmvj6rX7kUF0Ik1LG0Q6y?e=Rt9AsI

CommitMyZaru commented 1 year ago

I'm also facing this problem with mp4 h264 files streamed to WebRtc clients in a ffmpeg rtsp -> mediamtx -> browser setup. On some video files, despite explicitly removing any fancy h264 stuff (baseline profile) to ensure full browser webrtc compatibility, the stream freezes for a second or two, always at the same timestamps.

When I use mediamtx to "provide" the rtsp stream to another go-based webrtc server (that does no reencoding either), the freezing does not occur, so there's something specific about the way mediamtx handles the video/webrtc that amplifies this issue. Sadly this is very vague since I cannot provide the example videos for legal reasons, but I can try to provide any file/meta information that could be helpful.

aler9 commented 1 year ago

A complete explanation on why this bug happens is here: https://github.com/Glimesh/janus-ftl-plugin/issues/101

bluenviron / mediamtx