The first frame of WebRTC to RTMP conversion is very slow

[skill7899]

Describe the bug

The first frame of WebRTC to RTMP conversion is very slow

Version

5.0

To Reproduce Steps to reproduce the behavior:

srs.conf

listen              1935;
max_connections     1000;
daemon              off;
srs_log_tank        console;

http_server {
    enabled         on;
    listen          8080;
    dir             ./objs/nginx/html;
}

http_api {
    enabled         on;
    listen          1985;
}
stats {
    network         0;
}
rtc_server {
    enabled on;
    listen 8000; # UDP port
    # @see https://ossrs.net/lts/zh-cn/docs/v4/doc/webrtc#config-candidate
    candidate $CANDIDATE;
}

vhost __defaultVhost__ {
    min_latency off;
    tcp_nodelay off;
    time_jitter off;
    mix_correct off;

    srt {
        enabled     on;
        srt_to_rtmp on;
        }
    play {
        gop_cache off;
        queue_length 10;
        mw_latency 100;
    }
    rtc {
        enabled     on;
        rtmp_to_rtc on;
        rtc_to_rtmp on;
    bframe      discard;
    }
    http_remux {
        enabled     on;
        mount       [vhost]/[app]/[stream].flv;
    }

}

1.Execute the Docker command to run a container interactively with a pseudo-TTY,

docker run --rm -it -p 1935:1935 -v /srs.conf:/usr/local/srs/conf/srs.conf -p 1985:1985 -p 8080:8080 \ --env CANDIDATE=$CANDIDATE -p 8000:8000/udp \ registry.cn-hangzhou.aliyuncs.com/ossrs/srs:5 ./objs/srs -c conf/srs.conf

2.Execute the command

ffplay rtmp://localhost/live/livestream -analyzeduration 1000000 -fflags nobuffer

3. Open the specified HTTP URL in a web browser to access the RTC publisher page and initiate an RTP push by clicking the appropriate button or link.

4. for 10 seconds, after which FFplay should begin displaying the video feed.

5. Review the output log for information.

Expected behavior

1s can see the image in ffplay for fist frame

Screenshots

[xiaozhihong]

The RTC audio and video utilize a separate clock and require waiting for the sender report to synchronize. If this is not done, the timestamps for audio and video when converting to RTMP will be out of sync. This is the primary reason for the slow first frame. What is your scenario that requires RTMP to start quickly?

TRANS_BY_GPT4

[skill7899]

The RTC audio and video utilize a separate clock and require waiting for the sender report to synchronize. If this is not done, the timestamps for audio and video when converting to RTMP will be out of sync. This is the primary reason for the slow first frame. What is your scenario that requires RTMP to start quickly?

TRANS_BY_GPT4

H5 video anti fraud requires instant activation, otherwise the waiting time will be too long, which will greatly affect the user experience,H5 uses webrtc to connect to SRS, and the backend service pulls the RTMP stream from SRS. @xiaozhihong

[skill7899]

Are there any other methods to speed up this conversion process to achieve second-level speed, such as disabling sound? @xiaozhihong

TRANS_BY_GPT4

[winlinvip]

I believe we cannot fix this issue because it stems from a fundamental design choice. Specifically, WebRTC is not designed to prioritize the immediate display of the first frame. In online meetings, the rendering time of the first frame is not a concern, unlike in live streaming, where it is crucial. Therefore, there is a distinct design philosophy between WebRTC and live streaming.