Dropped many frames when use rtsp

[RedRackham-R]

Hi

I had the same problem as issue #393 .Everything works fine when I use VLC to pull and play the rtsp stream.But when I play with WebRTC I get a lot of frame dropped,log keeps printing

(h264_bitstream_parser.cc:187): Streams with pred_weight_table unsupported

(h264_bitstream_parser.cc:274): Failed to parse bitstream. Error: 1

(h264_bitstream_parser.cc:274): Failed to parse bitstream. Error: 2

in the verbose log. I'm not sure if this Error is affecting the dropped frames as this is my first exposure to WebRTC.

This is the log when playing ：webrtc_streamer.log

Hope this log will identify the problem

In addition, my English is very poor, so I need to use translation software, I hope it does not hinder the expression of the problem :D

[RedRackham-R]

update

When I use webrtc-internals to view the connection information of WebRTC,found to receive about 5-8 frames per second

like this:

So I re-checked the log and found the print info:

(video_stream_encoder.cc:1308): Same/old NTP timestamp (1666938371978 <= 1666938372103) for incoming frame. Dropping.

(video_stream_encoder.cc:1452): Adjusting allocation, fps = 8, from VideoBitrateAllocation [ [2000000] ], to VideoBitrateAllocation [ [1666667] ]

About a dozen times a second dropping. The fps of the video I pushed with FFmpeg is 24, which is basically consistent with the reception volume shown on webrtc-internals, so I think it may be related to the comparison of Same/old NTP timestamps, but I don't know how to deal with this problem yet.

[RedRackham-R]

update

This is the code of the video_stream_encoder part. Looking at the comments, it may be the dropping caused by capturing multiple frames at the same time? I don't understand the content of this part at all.

[mpromonet]

Hi,

It may be because your RTSP stream send B-frame.

Best Regards, Michel.

[RedRackham-R]

Thank you for your reply :)

After I cancel the B-frame with the FFmpeg -bf 0 parameter, the video plays correctly!

Does webrtc-streamer have the function of processing B frames?

[mpromonet]

Hi,

The implementation of webrtc use Web RTC SDK, you may try to ask for this in https://bugs.chromium.org/p/webrtc/issues/list.

Best Regards, Michel.

[RedRackham-R]

Thanks！ This is a great project！ :D

[zhuya1996]

Hello, has this frame loss problem been solved? Can you help solve it for a fee? Thank you, brother.

[RedRackham-R]

Hello, has this frame loss problem been solved? Can you help solve it for a fee? Thank you, brother.

@zhuya1996 This issue arises because WebRTC does not support processing B frames. When I use FFmpeg to stream, I resolve it by disabling B frames (-bf 0), and this ensures smooth playback.

[aiobofh]

@RedRackham-R how do i use webrtc-streamer and ffmpeg to do what you suggest? Do you have a code snippet?

I am running on ARMv7 and have a HDMI-capture device that introduce jerky stream and dumps verbose information about "Same/old NTP timestamp" often.

As far as i understand i need to start webrtc-streamer with some RTSP stuff that I don't understand and then make ffmpeg send its stream to that somehow? Help wanted.

[RedRackham-R]

@RedRackham-R how do i use webrtc-streamer and ffmpeg to do what you suggest? Do you have a code snippet?

I am running on ARMv7 and have a HDMI-capture device that introduce jerky stream and dumps verbose information about "Same/old NTP timestamp" often.

As far as i understand i need to start webrtc-streamer with some RTSP stuff that I don't understand and then make ffmpeg send its stream to that somehow? Help wanted.

My case is to push a local MP4 file to an RTSP server using ffmpeg and play it with webrtc-streamer. If you have other video sources, you can refer to the ffmpeg documentation to see if you can forward that video stream to the RTSP server or convert it to an RTSP stream in other ways. Below is my test process:

Step 1: Set up RTSP Server

Using rtsp-simple-server (now renamed to mediamtx, I am currently using the old version: v0.21.6).

Start the RTSP Server

./rtsp-simple-server

The default RTSP service port is 8554. For more details, refer to rtsp-simple-server.yml.

Step 2: Stream Local MP4 Video to RTSP Server

Use ffmpeg to stream a local MP4 video to the server.

FFmpeg Command

ffmpeg -re -stream_loop -1 -i test_video.mp4 -vcodec h264 -acodec opus -bf 0 -strict -2 -f rtsp -rtsp_transport tcp rtsp://127.0.0.1:8554/rtsp_test

Step 3: Configure and Start WebRTC-Streamer

WebRTC-Streamer Config File

Update the config.json file in webrtc-streamer to add the RTSP stream address.

{
    "urls": {
        "test_video": {
            "video": "rtsp://127.0.0.1:8554/rtsp_test",
            "audio": "rtsp://127.0.0.1:8554/rtsp_test",
            "options": "rtptransport=tcp&timeout=60"
        }
    }
}

Start WebRTC-Streamer

./webrtc-streamer -a -s127.0.0.1:3478 -tlxy:123456@127.0.0.1:3479 -Cconfig.json -N10 -o

---

After completing the steps above, I can play test_video.mp4 on the webrtc-streamer frontend page.