RTC2RTMP: First play FLV, then push the stream, only video appears, no sound

[winlinvip]

Describe the bug First, play the FLV file, then use WHIP to stream it. You may encounter a situation where only the video is visible, but there is no sound. Upon replaying, both sound and video are present.

Version SRS 6.0.95

To Reproduce Steps to reproduce the behavior:

1. First, play the FLV stream.
2. Then, stream using WHIP.
3. There's a high chance that only the video will appear, without any sound.

Expected behavior Play both video and audio.

Screenshots

Additional context Sometimes there might be audio without video, but most likely there will be video without audio.

TRANS_BY_GPT4

[winlinvip]

When the test fails, a log line will be generated. Refer to this action:

138 frames left in the queue on closing

During audio transcoding, FFmpeg Opus doesn't process a lot of data. 138 frames are about 3 seconds of data, and our case is about 5 seconds. The actual time given to process the stream for each case is about 3 seconds, so it times out.

This can also explain why, when playing FLV first and then pushing RTC, there is only video and no audio. This is because there are no audio packets at this time, so the player assumes there is no audio.

TRANS_BY_GPT4

[winlinvip]

Known issue, see https://github.com/ossrs/srs/pull/3883/files#diff-698428b8878a364edf15e43a4bdf6d5cb22f3e1f28faee253210a86196558bd6

# Please note that we must disable the ffmpeg-opus, as it negatively impacts performance. We may consider
# enabling it in the future when support for multi-threading transcoding is available.

[green-cats]

@winlinvip any updates?