search

ngraziano / SharpRTSP

A RTSP handling library
Other
557 stars 182 forks source link

Artifacts in H.264 video stream #85

Closed tepponen closed 1 year ago

tepponen commented 1 year ago

Hi,

We have encountered strange behavior with H.264 where increasing the "quality", resolution and bit rate values of the live camera stream results showing artifacts in the camera picture. Artifacts can be seen on the picture below. Frame rate changes doesn't have any effect on the issue.

window_via_nvr

The artifacts change place depending on how much information is on the image (how much details/changes is in the frame). When playing RTSP stream directly in VLC player, no such artifacts are seen.

When saving received H264 packets to the disk with the RTSPClient example, the artifacts are also visible on the VLC player.

Any pointers what to look for?

RogerHardiman commented 1 year ago

I know the artifact you are having. It usually means the keyframe (i-frame) is not getting through to the decoder.

Or it can mean you have network packet loss, and the green garbage is the point the keyframe data is getting lost.

Or it means the H264 receiver does not have a large enough buffer to read in a whole Keyframe.

Also when viewing in VLC it may be making a TCP connection (so masking any packet loss) but perhaps your other viewing software is running in UDP mode which will fail when a packet is lost.

Are you using the simple h264 encoder in sharpRTSP or have you replaced it with a proper H264 encoder?

tepponen commented 1 year ago

Thank you for the reply! Did some additional digging with this.

SharpRTSP receives H264 video in TCP mode. Received data is passed along to muxer (jMuxer library to be specific), which handles the video rendering on web browser.

Tested the RTSP stream in TCP mode in SharpRTSP (saving to disk example) and directly with VLC in TCP mode. In VLC video shows without any issues... Further checking the data flow to SharpRTSP with Wireshark revealed that TCP "Conversation completeness" is marked incomplete where the VLC data flow is marked as complete.

ngraziano commented 1 year ago

Hi, I'm not sure the "Conversation completeness" is meaningful because it follow if the connection is well close and that seems not to be the problem. Like RogerHardiman says, the video look like there is missing packet or missing end of packet. You should logs the rtp_sequence_number and check if some are missing. You can try to set the reception in UDP to see if the problem is the same (maybe in UDP the camera do note split I frame the same way) Check that in Process_H264_RTP_Packet you do not hit an unsupported agregation type (see the code in H264Payload)

It is hard to find the problem without an access to the camera.

tepponen commented 1 year ago

Checked the H264 RTP packet messages and no hits on unsupported aggregation types. Also did test where logging sequence numbers not matching to next one and this too didn't show up any anomalies.

I have created temporary access to the camera's RTSP stream with following URL: rtsp://test:password123@62.220.227.150:554/ISAPI/Streaming/channels/701

ngraziano commented 1 year ago

There was a bug in the RTP decoding. The padding flag was not handle (first time I have seen it used). The commit 4f48626 should solve the issue.

tepponen commented 1 year ago

Wow! Tested with the camera and does seem to do the trick. Padding being used never occured to me here.

I will run more tests with other cameras which produced similar issues, but this commit looks very promising! Will report back with findings.

tepponen commented 1 year ago

The commit https://github.com/ngraziano/SharpRTSP/commit/4f48626645c3cbc380260d1b36c02078f0ad9246 fixed this issue completely for all devices that previously had issue. Many thanks with this!