Closed kris6673 closed 5 years ago
On first sight I suspect omxplayer to be the cause. Did you try to run omxplayer standalone with exact same arguments as rpisurv uses?
--video_fifo 1 --video_queue 1 --live --timeout 60 --aidx -1 -o hdmi --threshold 0 --avdict rtsp_transport:tcp or without tcp --video_fifo 1 --video_queue 1 --live --timeout 60 --aidx -1 -o hdmi --threshold 0
What version of omxplayer are you using?
I did. When i ran it with the exact same arguments there was no stutter. Started rpisurv up and the stutter was there after about 10 seconds. Not sure, i don't have access the to the rpi atm. If it's any help, it's the default version that comes with a brand new install of Raspbian from 2 days ago.
Can you try to disable memory_usage_check: False. I had reports in the past that this was causing some issues.
It's really weird that the memory usage reported jumps around from almost nothing to almost everything in use. I never saw that before.
Is it a config setting? Yeah, its really weird. Just gobbles all the memory like it was cosplaying Chrome ^^
I turned memory_usage_check to false. Stuttering is still happening. This is what htop reports for memory usage when rpisurv runs, while the stuttering is still there. OMXplayer version is 061425a.
Small update: Even with no camera connected it still uses all the GPU memory.
Same here... tested it outside of your Script - but i'm not sure, if not the vcdbg reloc command is the cause of the problem...
PS: I've just made a rpi-update and installed the latest version of Raspbian.
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 29 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 31 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 30 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 30 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 33 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 32 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 29 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 33 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
7.8K free memory in 26 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 30 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 33 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 30 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 33 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 29 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
352 free memory in 3 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
352 free memory in 3 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 32 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 30 free block(s)
pi@raspberrypi:/usr/local/bin/rpisurv $ sudo /usr/bin/vcdbg reloc | grep 'free memory'
691M free memory in 32 free block(s)
Did you restart after setting memory_usage_check: False? I can not reproduce the errors you are describing.
I am running
root@raspberrypi:~# omxplayer -v omxplayer - Commandline multimedia player for the Raspberry Pi Build date: Mon, 05 Nov 2018 15:45:07 +0000 Version : 061425a [master] Repository: https://github.com/popcornmix/omxplayer.git
root@raspberrypi:~# lsb_release -a No LSB modules are available. Distributor ID: Raspbian Description: Raspbian GNU/Linux 9.8 (stretch) Release: 9.8 Codename: stretch
So, it seems there is an issue with the /usr/bin/vcdbg reloc command sometimes using up all the memory, can both of you confirm by stopping rpisurv. systemctl stop rpisurv. And run /usr/bin/vcdbg reloc | grep 'free memory' some times to see if it uses up the memory.
Thanks
This is the output of running sudo /usr/bin/vcdbg reloc | grep 'free memory' before and after the start of rpisurv.
Hi, I think we are running in circles here.
I can not reproduce the issue you are describing, so the only way to potentially solve this is for you to give me plenty of information :-)
We might be.
Sorry. If you know a convenient way of sending a 16 gb file, I could image the SD card and send it to you?
In your htop I see multiple streams?
Could you send me a video of the stuttering?
Alright. The multiple streams are child processes according to htop i think. gif of the problem: https://imgur.com/a/s7wQeLn
I am really puzzled. I see no reason why omxplayer should behave differently when started by rpisurv as when started manually. According to the info you provided me I see no issue witht the rpisurv code itself. Are you sure you started omxplayer manually with exact the same arguments? It could be that the stuttering you are describing is just because you have a low framerate. 15 fps can look like image is stuttering
I'm puzzled myself, it worked perfectly for 2 weeks, sat around unpowered for 1 week and then the stuttering began. The pi has always been shutdown with a shutdown command, not just pulling the power. It's the exact same arguments on omxplayer, iv'e checked it far too many times. When i look at the stream in VLC it's very clear that it's not the stream being 15 fps that's the problem. No stuttering at all when playing from VLC. Not sure what to do now, got any other ideas or should we just close this issue and call it a day?
Are you also running that VLC on from the raspberry pi? But then again you are running omxplayer from the raspberry pi too. Maybe a hardware problem, do you have another pi to try, or even another camera? Or maybe try to substreams of your camera
I'm out of other options
Nope, from a Windows PC on the same network. Tried with other camera and that stuttered too. Tried 3 substreams, Still stuttered.. :( Since last time i have installed displaycameras on another SD card and that runs like a charm, not stuttering at all, even if the features leaves a bit to be desired comming from rpisurv. Where in the code do you determine if you need to make another stream/omxplayer window? Those "child processes" might actually be full on streams as you thought earlier. Grasping for straws here. Sadly i don't have another Pi to test on, we would know real fast if it was a hardware problem then.
The code managing the worker processes is here: https://github.com/SvenVD/rpisurv/blob/master/surveillance/core/worker.py
What is your config file? You are using one screen right? No rotation?
You are mentioning it did work for 2 weeks without stuttering? What was changed, besides the pi being unpowered for a while.
Thank you. Nothing out of the ordinary, i'll upload it when i get access to it on monday. 1 Full HD screen, no rotation. Nothing at all, we changed the url in the config but that's all. It just sat there and when we powered it up again the issue was there.
Just an idea, there aren't other sources pulling streams from the same camera are they? Cameras are limited to a certain amount of streams, more expensive cameras can handle more simultaneous streams.
Nope, both camera and rpi are only used for eachother.
Didnt find the solution and not gonna spend more time on it. Thanks for the help anyway Sven!
I have the exact same setup and problems. Unable to find the solution either.
Hi there Thanks for making a great program! I'm having some issues though. I'm running a raspi 3 setup with 1 Vivotek 1920 x 1080 15 fps camera in full screen. I'm experiencing stutter when the program recieves a keyframe/ intra frame it hangs for about 250-500 ms. I have tried lowering the resolution and framerate. Same problem. I tried running the stream directly with omxplayer, did that just fine, no stuttering/hanging at all. The stuttering isn't there for the first 10 sec from startup, then ramps up from there. I have tried reinstalling the program and Rasbian, issue persists.
I'm fairly certain that it's a memory usage issue, since it does not matter how much GPU memory I assign it in /boot/config.txt always throws GPU mem errors. From 64 to 512 , throws the same error. Currently i have 512 memory for the GPU. If you need anything else, please let me know. Thanks!
-Kris6673
These are the errors from /var/log/daemon.log:
Log from /usr/local/bin/rpisurv/logs/main.log: