RTSP decoding errors in 1.26.4 [$450 awarded]

[mastertheknife]

I have created this in order to centralize the discussion and the workarounds around the problem that some users are experiencing after upgrading to 1.26.4 The problem is mentioned in few threads in the forum, such as: http://www.zoneminder.com/forums/viewtopic.php?f=30&t=21550 http://www.zoneminder.com/forums/viewtopic.php?f=30&t=21516

The $450 bounty on this issue has been claimed at Bountysource.

[dvarapala]

Guys,

While 1.26.4 broke something for some people, it fixed things for others. My Dahua RTSP camera was dead in the water under 1.25.0 but is now working great under 1.26.4. Please keep this in mind while you work towards the ultimate solution that works for everybody. Thank you for all your hard work - ZoneMinder rocks!! :)

[chippy99]

I have been trying to figure out what the problem was and have done several recompiles of both ZM and ffmpeg and I have now got a 1.26.4 build that works with mpeg over rtsp. Will try to discover what the fix was tomorrow.

[chippy99]

I got a bit confused with all the recompiles and also trying different versions of files. The problem is in zm_rtp_source.cpp. If I compile 1.26.4 with zm_rtp_source.cpp from 1.26.3 the decoding errors disappear, but I cannot figure out what is wrong with latest zm_rtp_source.cpp, its a bit beyond me.

[POKKAHOH]

As I answer in forum

"The problem may be that it is your camera offers a choice of several implementations of the flow ... in version 1.26.3 when trying to take a fragmented NAL was an error and was used mjpeg."

See issue https://github.com/ZoneMinder/ZoneMinder/pull/174#issuecomment-25602723 and other comments

[chippy99]

Sorry, I dont really understand your reply. Are you saying it is a problem with my cameras or their setup ? They are fairly mainstream manufacturer not a chinese clone.

[POKKAHOH]

:) 1.26.3 never worked with RTP, if there was a response with the fragmented NAL. Was possible to work only with mjpeg.

1.26.4 is able to collect the fragmented NAL, the work was made possible by RTP.

But it uses ffmpeg library libavcodec version 54 and above.

The links have pictures of the screen which shows camera settings and picture with these cameras.

[mastertheknife]

@POKKAHOH

mjpeg is fine if thats the only thing supported. Falling back to mjpeg is definitely better than leaving the user without any stream at all. If you want to notify the user that his stream is using mjpeg right now, can use Warning() for that. Leaving the user with a non-working camera is terrible.

[POKKAHOH]

Of course, this must be done ... just I'm a noob in programming multimedia. And this -> to be continue :)

[chippy99]

I feel a bit let down by the way ZoneMinder is going. Why was this code allowed into 1.26.4 when it breaks mpeg and only makes provision for h.264 ? I see a lot of activity going on for good stuff in the future but nobody seems to be concentrating on the current core zm, which if a new user installs it is broken.

[knight-of-ni]

The trouble is that the latest changes to the RTSP code fixed some cameras and broke others. We've been discussing the status of this issue almost daily. The issue has not been quickly fixed, not because we don't want to work on the issue, but because we aren't sure yet exactly how to fix it. The problem is compounded by the developers lacking a camera that shows the symptoms of this issue.

You can expect this issue to be fixed in the next release.

Some things you can help with this issue are:

• We need more data regarding this issue. A wireshark session might help at this point.
• Provide access to your camera
• Get on irc and help us troubleshoot. We are typically most active in the mornings (central/eastern time)

[kylejohnson]

@knnniggett, thanks for jumping on.

@chippy99

Why was this code allowed into 1.26.4 when it breaks mpeg and only makes provision for h.264 ?

We did not have cameras which exhibited this issue. I personally tested the code that introduced these issues with my own cameras which support RTSP, and did not encounter any issues. If we had access to more cameras, things like this would certainly happen less frequently.

I see a lot of activity going on for good stuff in the future but nobody seems to be concentrating on the current core zm, which if a new user installs it is broken

I agree. We need more, better, automated, testing.

With that said, we need help. The first thing that I can think of, which doesn't require super technical skills, is someone that can get more comfortable with the Travis CI build system. Specifically, a hook for each commit to build ZM (which already happens) and then insert test cameras into the system and then check error logs for issues. This is something that I was working on previously, but could use help with.

We'll have this fixed as soon as we can.

[chriswiggins]

More importantly, we need a diverse range of different cameras with public access to them!

[chippy99]

I can help with the cameras, need to do a bit of configuring on my router, its a bit late now (2:50). Hopefully will have it sorted in the morming.

Am totally unfamiliar with Travis CI build system but I am a quick learner and will do some research on it over the weekend and try to help where I can.

Am impressed by quick responses from you all. Want to help wherever I can because ZM is basically a great project. Will try to get onto irc tommorow.

[POKKAHOH]

I see two problems 1 I need access to the "broken" camera (or at least the log from tcpdump) 2 the time difference I GMT +4, most GMT-4 and the main dialogue is via e-mail

I repeat - my edit for ONLY the correct assembly of a fragmented NAL and reordering packets when receiving rdp flow. ALL, nor work with the database, nor connect the camera - nothing more.

The only bug that I know about - not checked version of ffmpeg libraries. The question of whether there is enough to put a conditional compilation, which was asked in IRC? remained unanswered. It can be seen lost in technical communications server. All those wishing to can send diff file between the master branch and edits that are made ​​to me.

[chippy99]

Have setup a Vivotech IP8133W that should be accessible to all connection details are below. The stream on live3.sdp is mpeg4 and live.sdp is h264. It will give a black picture between about 23:00 and 08:00 GMT because no lights are on in the building.

[chippy99]

Here's another, its a Vivotech 8130W. It only does mpeg.

[mastertheknife]

@chippy99

I have connected to both streams and they are working for me. Perhaps i am doing something wrong?

[chippy99]

Neither of these work for me on 1.26.4. I get the error decoding frame errors unless I revert zm_rtp_source.cpp back to 1.26.3 version.

[connortechnology]

Please try my crappy camera (Internet Eye MPEG4).

at www.connortechnology.com:554

Not sure if the path matters. ffmpeg works without a path, but from what I've read, you can use /cam1/mpeg4 or /cam1/h264 to get different stream types.

I have not got this working with the rtsp code in zm. However ffmpeg works.

admin:password@

[mastertheknife]

@chippy99 I'm using сurrent git. I will try with stock v1.26.4 (git checkout v1.26.4) and report back.

[mastertheknife]

@chippy99

My bad, i just noticed i actually do receive those errors, but not for all frames. However, the stream and monitor definitely work.

[chippy99]

Ok, thats good. I do get a picture, but not all the time, because some frames make it through, but most fail.

[mastertheknife]

The issue is that decoding the frames (avcodec_decode_video2) is failing for few frames now and then. I believe the issue is something with the frames sequence. It seems that the sequence is lost about every ~10 frames (for chippy99's stream) and then the decoding fails because a frame is missing, until it reaches a keyframe and continues from there.

[chriswiggins]

I've had mixed success with storing the last received key frame and using that if one is lost. Of course with lots of testing it doesn't work for all scenarios but it's a tactic we could try and even see if we can improve it. How does VLC and ffmpeg handle these issues?

Sent from my iPhone

On 10/11/2013, at 7:46 am, "Kfir Itzhak" notifications@github.com<mailto:notifications@github.com> wrote:

The issue is that decoding the frames (avcodec_decode_video2) is failing for few frames now and then. I believe the issue is something with the frames sequence. It seems that the sequence is lost about every ~10 frames (for chippy99's stream) and then the decoding fails because a frame is missing, until it reaches a keyframe and continues from there.

— Reply to this email directly or view it on GitHubhttps://github.com/ZoneMinder/ZoneMinder/issues/221#issuecomment-28133920.

[mastertheknife]

They handle it the same way as we does. If the stream is unreliable (UDP for example) and a frame was lost, wait for next keyframe. If we connect over TCP, no frames shall be missing, so it shouldn't happen. I think there might be a problem with the internal ZM rtsp code and the way it parses data.

[chriswiggins]

So is the solution to keep the last available image in a buffer in case this happens? I mean the image will be still for approx 0.5seconds (depending on key frame frequency) but that's better than dropping it completely?

Sent from my iPhone

On 10/11/2013, at 7:54 am, "Kfir Itzhak" notifications@github.com<mailto:notifications@github.com> wrote:

They handle it the same way as we does. If the stream is unreliable (UDP for example) and a frame was lost, wait for next keyframe. If we connect over TCP, no frames shall be missing, so it shouldn't happen. I think there might be a problem with the internal ZM rtsp code and the way it parses data.

— Reply to this email directly or view it on GitHubhttps://github.com/ZoneMinder/ZoneMinder/issues/221#issuecomment-28134114.

[POKKAHOH]

@chippy99, in my test zm server i can see you bouth cam... that`s one (in both stream - same picture)

that`s second (some times - became black square)

differents between zm_rtp_source.cpp - only collect the fragmented NAL...

im really cant understand what not work and why...

[chippy99]

@POKKAHOH If you look at the console log you will see many frames are being dropped

[POKKAHOH]

@chriswiggins for you question https://github.com/ZoneMinder/ZoneMinder/issues/221#issuecomment-28134003: ffmpeg s show broken pictire (like "rain window"), VLC-1.X -same ffmpeg, VLC-2.0 (see class RTSPClientVlc ) use some libs ( from live555) to reorder frames.

[POKKAHOH]

The assumption that the error is caused by the wrong type of decoding the packet was wrong.

Now I think I dropped frames these are the "floating frame"/"rain window" in ffmpeg That is, in fact now working with RTSP uses only basic frames. The rest are simply discarded ...

We can ignore error, we can correct the source code for the INFO of a WAR or informational message (see http://www.zoneminder.com/forums/viewtopic.php?p=82213#p82213)

But is it right to be rewritten work with the flow, may be as already suggested - using curl.

[kylejohnson]

Added $50 to the bounty.  Any takers?

[POKKAHOH]

I think I'll do that. Free of charge. It is a matter of honor.

[kylejohnson]

Hi everyone. I've just upped the bounty again. This issue is becoming high impact, and is impacting some of my clients.

I've experienced the same issues as others reported (error while decoding frame) with grandstream gxv3611 cameras.

[ebarnard]

This has been annoying me quite a bit. I've managed to get rtsp working without errors for all my cams and the three listed here. There's still quite a big detached frame effect on some mpeg4 cams and it fails to handle some h264 frames correctly (they're not video frames though - 8 and 23 bytes and each identical). Can anyone else confirm if this does/doesn't work. Repo at https://github.com/ebarnard/ZoneMinder/tree/rtsp-fix

[chippy99]

@ebarnard I just installed on my home system and it is looking good with mpeg4. I only have one cheap Avermedia IP camera on it but no errors in last 20 mins. Will try it on live system tomorrow and will be able to test both mpeg4 and h264.

So far so good :>)

[chippy99]

Loaded onto my live site and there is a problem with bottom half of image constantly stretching up and down, this is using mpeg4. You should be able to view a video of this at http://bttc.co.uk/zone/rtp.avi

[ebarnard]

That's strange. Did it happen on 1.26.3?

[ebarnard]

@chippy99 It looks like I was cutting off the last part of a fragmented mpeg4 frame. The latest commit fixes the problem on a few public cams I've found so let me know if it works for you.

[kylejohnson]

I'll test your code as soon as I can with Axis and Vivotek cams.

Ed Barnard notifications@github.com wrote:

@chippy99 It looks like I was cutting off the last part of a fragmented mpeg4 frame. The latest commit fixes the problem on a few public cams I've found so let me know if it works for you.

---

Reply to this email directly or view it on GitHub: https://github.com/ZoneMinder/ZoneMinder/issues/221#issuecomment-29525504

[chippy99]

Just tried it on MPEG4 with 2 diferent models of Vivotech cameras and it is looking good :>) Will try h.264 later.

[chippy99]

When set to h.264 I get some error while decoding frame mesages in the log.

[ebarnard]

@chippy99 That was bugging me for a while but it turned out to be due to sps and pps frames not being handled properly and should be fixed in the latest commit. I haven't managed to find a test camera that shown an error so hopefully this should fix this issue.

[chippy99]

Yep error has now gone. Presume all is ok, cannot see a picture because place is in darkness at the moment

[POKKAHOH]

@ebarnard - all work. I can see kylejohnsons cam and chippy99s Vivotech IP8133W (both h.264/mpeg) cam without error.

Chippy99`s Vivotech 8130W show errors

in my hikvision it work same as my patch

[POKKAHOH]

Summarize a bit ... now we have no problem with image decoding. We need to do the normal flow control RTP / RTSP This conceptual question I see the following ways of development

1. Use live555 (directly or through videolan library)
2. Use Curl library
3. Using ffmpeg library
4. Writing (revision) own class

Which way will move further development?

[ebarnard]

The errors on the Vivotech 8130W are due to ffmpeg not having recieved either a keyframe or stream parameters (I forget the proper name for these). I'm guessing this is a bug somewhere in the RTSP code or the camera not storing the last keyframe. It could be "fixed" by not displaying decode errors unless at least one keyframe had been received or a sufficient time has passed but since I, like you, want to replace the current RTSP implementation it seems a bit of a waste of time to do - but I'll do it if you really want.

As for what to do next, my thoughts are below.

libvlc - This would be very simple to implement, in fact I've almost got it working. The problem is that there is no public api for getting undecoded packets from the demuxer or for directly transcoding a stream which would cause problems when trying to implement video storage. Nonetheless as it's so easy to do I think we should implement it if only to provide basic support for most protocols (e.g. would fix #253).

live555 (liblivemedia) - A little more complex to implement but I think this is the way forward. As far as I know it's fully standards compliant. The only issue is both this and libvlc don't support ipv6 (and likely won't for some time).

ffmpeg - As far as I know this doesn't support digest authenitcation and as it handles everything from io to demux opaquely I don't think there would be a simple way to add it.

write our own - Given the above choices I think this is something we want to avoid. The same goes for adding curl to the existing implementation (we'd still have to implement RTP).

[ebarnard]

The other option would be something like JRTPLIB for RTP and then curl for RTSP. JRTPLIB isn't maintained any more and we'd have to provide implementations for special stream types (e.g. H264 NALs) although there might only be a few (or even one) of those. On the other hand it does support IPv6 which I assume would be very difficult to add to live555 (given that nobody working on VLC has yet done it). I actually quite like this idea.

Also unless anyone has any further problems/suggestions with my, albeit temporary, fix I'll submit a pull request.

[chippy99]

All my cameras are working ok. I get decode errors for frame 0 on startup with mpeg4 but have always had these so its not a problem.

[POKKAHOH]

And so, I vote for adding another dependences - vlclib :) There is builtin live555 and camera support "just like in vlc"