RTSP Streaming Errors

[74ls04]

Issue description

I'm creating this issue to discuss the degradation of stream quality when the camera is working in high exposure conditions like being pointed outdoors in daylight. I believe this encapsulates issues #353, #81, and #44.

Steps to reproduce the issue

1. Point camera outdoors during bright daytime
2. View RTSP high stream in any viewer like VLC or using ffmpeg with ffplay

Expected behavior

• The stream plays with no glitching similar to how it does in the Yi app.

Current behavior

• RTSP high stream shows occasional but consistent artifacts (glitching) with no motion and major glitching with any major motion in the frames
• RTSP high stream shows ghosting when there is significant motion in frame (like a car driving by). Often when a car passes by moving quickly it will be shown in the frame, the video will appear to freeze, then the car will reappear in an earlier position before then jumping ahead. Sometimes the previous position of the car will show boxy glitching. It seems as if the frames appear out of sequence at times. Unfortunately I can't provide video in the ticket as that would possibly reveal my home.

Here is an example of someone showing up in two places at the time:

Additional details

• I verified this behavior using VLC, ffmpeg with ffplay, motioneye, and a Golang app RTSPtoWeb. It was present on an i3, and i7 computer, and in a Ubuntu VM.
• It is not a network bandwidth issue, I tested with only a single hop between the computer and router with a total distance of about 4 meters. Other cameras in the house that are 10x further away from the router don't show this issue.
• I tested both a y20ga and y25ga camera and they both showed this issue when pointed out the same window.
• I measured the camera voltages and they are all within spec

I performed a test by doing an RTP stream analysis with Wireshark with the camera pointing outside and then putting a dark sheet in front of the camera without moving it.

In dark conditions, the stream analysis was perfect with no errors as shown in the screenshot below.

When I removed the dark sheet, the stream analysis showed lost packets and sequence errors.

Interestingly, in the light conditions the analysis shows tons of packets with no markers while the dark one showed regular markers. Could this be why the decoding is being messed up?

Here is what ffmpeg sees:

Finally, I verified there were no errors being displayed on the camera -- the debug seemed normal.

And just to humor myself (I have no idea what I'm doing in this regard) I compiled rRTSPServer with an increased buffer size of 288000 and also increased OutPacketBuffer::maxSize to the same with similar results. I even compiled with the latest live.2022.04.26.tar.gz and that didn't fix it. Beyond that I don't fully understand the program or what's going on in /dev/shm/fshare_frame_buf.

Here is what the errors look like in ffmpeg:

In RTSPtoWeb

It's pretty interesting that without motion the glitching is happening in the bottom third of the screen for all the programs so it's like a systematic error.

I'm not a video person so unfortunately I can't be of much help without brushing up on the terminology and concepts. I do know it's not a network issue and all the RTSP clients are seeing the issue but how they display it varies based on their algorithms.

[roleoroleo]

There could be 2 explanations in my opinion: 1 - The cpu or the hw codec is not enough and, when you have big images (in terms of bytes), the cam doesn't have enough resources. 2 - Daylight means big frames (especially the i-frame) and there may be some small buffer in the chain (the fifo or inside rRTSPServer.

1 - You could try to switch off all unnecessary services (especially snapshots), disable audio and remove swap space (if you are using it). 2 - Are you able to check the size of the frames? Did you test the rtsp alternative server?

[74ls04]

Interestingly my alternate rtsp server isn't working - just connecting and disconnecting.

I think you're right about the big frames.

1. Disabling the unnecessary services doesn't seem to have an effect.
2. I used ffmpeg to capture the frame sizes and there is definitely a difference between an outdoor and indoor camera. For the indoor one the frames that aren't i-frames are smaller than the theoretical 1446 byte max RTP payload. For the outdoor one they are larger. Also the i-frames are quite large and from my reading if any packet of those are lost then the whole thing is lost. It seems to be a point of discussion in the live555 mailing lists [1] [2]

Here is the raw data indoor_data.txt outdoor_data.txt

I left the ssh session open for a while and caught these warnings so the server is seeing something too

[roleoroleo]

The bigger frames are about 160 KB. I will check if there is a bottleneck in the chain.

About alternate server, try to configure it from the web interface. It needs h264grabber to run.

[74ls04]

I had used the web interface to start the alternate server. The alternate server is not working for both my y20ga cameras (0.3.0 firmware). It also only worked once on one of my two y25ga cameras (0.2.8 -- I'm having that issue of it not upgrading and haven't had time to address it) -- the stream wouldn't start but on one of my many attempts it started. There are no errors logged on the camera, just those client connect/disconnect messages after about 5-10 seconds.

[Raeprice]

Was this ever resolved OP? @74ls04 Having same issues

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.