Latency, buffer of rtsp IP camera stream, causing unacceptable latency?

[Handssolow]

(To those working on omxplayer (and the RPi project) many many thanks.)

The RPi and omxplayer are being used to display IP security camera streams, but its low power means it cannot keep up in real time with the input of even a modest data stream without buffering the stream. The result is latency, I measure it's about 1 minute for every hour, unacceptable in a security context.

If I disconnect the Ethernet cable from the RPi after a couple hours, the screen doesn't blank or freeze, it runs on to display the buffered IP camera images, equal in time to the measured latency, nearly 2 minutes after 2 hours.

With video, a film stream say, omxplayer works, buffering and latency is not an issue providing there is no loss, or choppiness watching the film.

I seems that I have no control over this buffering, perhaps with IP cameras all I can do is not display HD and reduce the frame rate, bitrate, resolution etc to what the RPi can manage to keep up with. Buffering of a normal IP camera stream causes unacceptable latency.

Latency also happens using my XP netbook (1.6GHz Atom) but it's more acceptable, from 1 second of latency it increases to 7 seconds over 4.5 minutes, then the image displayed skips forward in time, latency is back to 1 second. True I've missed a few seconds for the sake of keeping up to time.

My Ubuntu Desktop from the LAN is powerful enough to display camera streams under VLC with latency always less than second.

Back to the RPi, I've found no configuration of omxplayer which prevented this buffering, video_queue, video_fifo etc.

How might omxplayer flush the buffer, perhaps every second, to keep latency low, yes at the cost of not displaying all of the streamed data, omxplayer would then display IP cameras in HD, in real time and low latency.

RPi 3B+ 1.4GHz GPU 500Mb

omxplayer - Commandline multimedia player for the Raspberry Pi Build date: Wed, 05 Jul 2017 16:40:39 +0100 Version : 5a25a57 [debian] Repository: git@github.com:XECDesign/omxplayer.git

Linux raspberrypi 4.14.61-v7+ #1133 SMP Fri Aug 10 11:04:43 BST 2018 armv7l GNU/Linux

Edit: I corrected an error. The latency of display with Ubuntu VLC was less than 1 second.

[adutra22]

We're using another program (rpisurv) on top that calls to OMXPlayer that handles up to 9 feeds on the screen at once, with only a couple seconds of latency that we've seen thus far.

[popcornmix]

It's pretty hard to give advice given the information.

but its low power means it cannot keep up in real time with the input of even a modest data stream without buffering the stream

not really accurate. The Pi can play BluRay (around 40Mbit/s) video over network without buffering.

Need to know what the codec/resolution/bitrate of the security camera streams is. Also upload the omxplayer.log file obtained from omxplayer -g to a paste site.

[Handssolow]

Regarding the cctv camera latency (1 minute per hour) using RPi omxplayer, with the two Chinese Leftek PTZ cameras from Amazon. A similar Anpviz PTZ camera is slightly better (30 seconds per hour). Disconnecting the ethernet feed from a camera, omxplayer plays out the latent buffered rtsp feed, for a minute say then the monitor goes blank then omxplayer terminates. I have spent many hours looking for a way to eliminate this latency issue. I have purchased two other IP cameras, a Hikvision 4MP PTZ and a Hikvision 3MP fixed focus, thankfully omxplayer displays the streams without significant latency, though using an RPi I have to accept a display of somewhat lower resolution than the cameras are capable. So in short, I have an omxplayer- cheapish Chinese IP camera issue. I have no idea why Rpi/omxplayer is able to process a Hikvision camera h264 rtsp camera stream OK but needs to process and/or buffer the stream from my Leftek or Anpviz cameras.Incidentally the Hikvision fixed focus camera was cheaper to buy than the Anpviz PTZ! Currently I have three IP cameras I cannot use with omxplayer because of latency.

If I could stop omxplayer buffering the rtsp stream it might keep up to real time.

On Tue, 25 Sep 2018 at 14:59, popcornmix notifications@github.com wrote:

It's pretty hard to give advice given the information.

but its low power means it cannot keep up in real time with the input of even a modest data stream without buffering the stream

not really accurate. The Pi can play BluRay (around 40Mbit/s) video over network without buffering.

Need to know what the codec/resolution/bitrate of the security camera streams is. Also upload the omxplayer.log file obtained from omxplayer -g to a paste site.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/popcornmix/omxplayer/issues/647#issuecomment-424354150, or mute the thread https://github.com/notifications/unsubscribe-auth/AfaeX48y_PGJHLhaqioli4oWsNhsIpt2ks5uejazgaJpZM4WOQfJ .

[Handssolow]

I would like to resolve why omxplayer is not working with two makes of IP cameras I have, latency is the issue (11 seconds over 5 minutes), another make of IP camera works reasonably normally (latency < 1 second over 5 minutes). A 3B+ RPI is directly connected to an IP camera here, not on the LAN. The PSU is rated at 2.5 amps. If I disconnect the camera, the RPi continues to play out the latent delayed stream images that it has been buffering. A restart of omxplayer resets to low latency, after a reboot it might even be lower still albeit temporarily.

Edited test results below are from 5 minute runs from two makes of camera. The camera 1 (2MP) has significant latency, camera 2 (3MP) doesn't. Both are set to display the lower resolution sub stream.

cam1_omxplayer.log https://pastebin.com/ETzpZ17R cam1 terminal output https://pastebin.com/PmzhLvs7 cam2_omxplayer.log https://pastebin.com/ptdwTUZS cam2 terminal output https://pastebin.com/8eDyp7va

The original size of an omxplayer log after 5 minutes was 3.6MB before an edit for Pastebin

[Handssolow]

Sorry I didn't intend to close this issue

[jehutting]

Have you tried it without the --live option and with the --aidx -1 option? The later disables the audio stream. Cam 1 delivers audio whereas cam 2 doesn't.

[Handssolow]

Thanks for your reply. I have no significant latency at any time from camera1 on a monitor connected to the NVR or either using VLC on a LAN connected Ubuntu desktop PC. RPi omxplayer without the --live option and with the --aidx -1 option, yields these latency results- Latency initially 2secs, 5mins/7secs, 10min/15secs, 20mins/20secs, 30mins/26secs, 40mins/31secs

Adding --live and --aidx -1, the first run results- Latency initially about 1 sec, 5mins/2secs, 12mins/2secs, 20mins/4secs, 30mins/4secs

Second run with same omxplayer settings- Latency initially 2secs, 11mins/9secs, 20mins/13secs, 30mins/20secs, 40mins/27secs I left omxplayer to display camera1 for 2 hours, the latency became 3 minutes.

I have not seen info about the terminal output if including the omxplayer option -s / --stats, pts and buffer stats. Towards the end of the last run of 2 hours, I believe I saw, M 101088, v 10.51s, 4800k/, 4800k, A: -10155.25, 0.00s/, 0.00 CV: 1844k, CA OK.

Any informative explanation of these results and my previously posted log files would be appreciated.

[premultiply]

Growing or shrinking latency means that the clock recovery on decoder and PLL adjustment on output does not work (correctly).

[Handssolow]

premultiply, I am not able to appreciate the significance of your last comments. Are you suggesting that omxplayer and camera1 are in some way not sharing clock information and so in time drift out of sync? I am still mystified by this fault, is it camera1, omxplayer or more likely some incompatibility between the two. To recap camera1 (Leftek) has unacceptable latency using omxplayer for display camera2 (Hikvision) works normally with omxplayer -no latency issue camera1 works OK with the NVR's monitor and VLC Ubuntu Desktop -no latency camera1 on XP VLC Netbook, latency cycles over 5 minutes, drifting to 7 seconds then down to 1 sec

When omxplayer has been displaying camera1 then the network cable disconnected from camera1 to the RPi, omxplayer continues to display the camera1 stream for the time of the latent period. Presumably the RPi has been buffering if unable to display the h264 image stream in real time.

Currently I am thinking that I am needing omxplayer to make up for some deficiency in the quality of the rtsp stream from camera1, time syncing etc. as omxplayer does work OK with camera2.

[Handssolow]

There seems some similarity here to issue #621, in that despite setting the --live option buffering of the rtsp stream occurs. I am unclear if the buffering experienced with camera1 is the cause or effect of my latency issue, ie if there were no buffering in omxplayer would there no latency issue? As camera2 works without significant latency, this points to some missing or corrupt timing element in the stream from camera1 which is absent in the rtsp stream from camera2.

[jehutting]

Thanks for the --live --aidx=-1 try-out. The first run with --live --aidx=-1 looked promiseful but in the second one omxplayer seems to fall back into the large latency.

From cam 1 logging

                                                 vv---clock speed
09:59:26 T:1613651240   DEBUG: Live: 2.18 (2.24) S:1.010 T:0.70
                              A:4260000 - M:2016286 = 2243714
                   audio pts ^^^          ^^mediatime
10:04:40 T:1928184623   DEBUG: Live: 1.14 (1.11) S:1.001 T:0.70
                              A:318780000 - M:317668794 = 1111206 ===>>> 1.1 seconds
   The video pts at this mediatime is   video pts= V:328346667 
                        ==> 328346667-317668794 = 10677873   ===>>> 10 seconds

Seems to me that omxplayer is not speeding up (long/big) enough to catch up with the stream.

Speeding is done (in omxplayer.cpp) by

            m_latency = m_latency*0.99f + latency*0.01f;
            float speed = 1.0f;
            if (m_latency < 0.5f*m_threshold)
              speed = 0.990f;
            else if (m_latency < 0.9f*m_threshold)
              speed = 0.999f;
            else if (m_latency > 2.0f*m_threshold)
              speed = 1.010f;
            else if (m_latency > 1.1f*m_threshold)
              speed = 1.001f;

Maybe as a new experiment, you could try to set --threshold=0 for cam 1, and try --threshold=2 with cam 2. The latter is an attempt to start (resume) the clock in the same point of time as cam 1 ( cam 1 setStartTime 1.880000 whereas cam 2 setStartTime 0.840000).

By the way Leftek camera, do you have a link where you bought them? It seems that that brand is not sold in the Netherlands, or maybe it is rebranded. Or maybe someone knows a cheap one (not necessarily a PTZ camera) sold here and with the same issue :-).

[Handssolow]

The Leftek POE PTZ IP Camera Mini Dome 2MP is currently available from Amazon UK https://tinyurl.com/yboewme6, they have a similar 5MP camera, both it seems also available from Amazon.com. I bought a second Leftek 2MP PTZ camera before I noticed the latency issue, in all respects it behaves like the first. Leftek, potentially a reasonable good camera for the money but useless using the RPi for cctv security display.

I have another Chinese IP camera bought on Amazon which is only slightly better than my Lefteks, in contrast my Hikvision IP camera works with omxplayer.

Tests adding a threshold settings had little positive effect, camera1 is Leftek, camera2 is Hikvision. Camera1 --threshold=0, start/1s, 5m/7s, 10m/13s,15m/20s, 20m/25s, 30m/36s Camera2 sub-stream --threshold=2, start/<1s, 5m/1s, 10m/1.5s, 15m/1.5s, 65m/<1s Camera2 sub-stream no threshold set, start/<1sec, 30m/<1s, 60m/<1s

Camera1, from info on ipcameratalk I tried the option --lavfdopts probesize:25000, gave after 84m/97s.

Despite setting --live option in omxplayer, buffering of the Leftek rtsp stream still happens, this causes latency. In some way the stream from the Hikvision camera can be displayed in real time and doesn't need to be buffered by the RPi.

[jehutting]

Thanks for the Leftek url and the result of the threshold test even if I am none the wiser.

I am out of ideas...

[dhsemp]

I’m also seeing serious latency/buffering issues using UniFi Protect RTSP on a Rpi3 (fully updated) using omxplayer with the ‘live’ option.

Has anyone figure out the cause of the buffering or how to properly debug? The RTSP streams will be delayed by minutes after 12-24hrs.

Thanks in advance for the help!

[oceaness]

Just to add, I'm experiencing the same issue.

Brand new Raspberry Pi 3 Model B+ GPU_MEM=256 Ubiquiti Unifi Video controller RTSP streams --live flag

1 x 1080p camera feed on screen, 1 x 1080p camera feed off screen, rotating between the two every 10 seconds and they were approx. 40 seconds behind after 21 hours.

Current work around is to restart the feeds every couple of hours, but would prefer not to have to do this.

[capqwerty]

I also have a similar experience with Raspberry Pi 3 Model B+ GPU_MEM=320 Ubiquiti Unifi Video controller RTSP streams 4 feeds all set at the low res. Users report 10+ minutes when running for a couple of days. If we restart it all looks good again.

[alexanderbont]

Same problem here, tried it with an Raspberry Pi 3B Plus and a Ubiquiti G3 Flex camera. RSTP stream at medium and high quality, both the same result. After a while, it's giving a delay on the stream.

Now I wonder, would the new Raspberry Pi 4 get rid of the delay, or is it just a software problem with omxplayer?

[popcornmix]

It's a software rather than hardware issue. Handling live streams is quite different to playing from a file and onxplayer was designed for file based playback.

Have you tried with VLC? That is hardware accelerated in recent raspbian images and may handle this case better.

[Wazirri]

Same thing happens to us too. That is why we also try VLC and ffplay.

VLC and ffplay uses more cpu than omxplayer.

Also another issue that sometimes omxplayer stops.

This happens when some of chinese camera brands. When cameras time set, stream stops. VLC also stops, ffplay keeps playing.

Maybe you create that player for file, it also works good with rtsp but just a simple things. Can you help to solve those issue?