Videos record fast

[PicoPi]

I have a RPI 3 and Pi Camera running your latest version 6.4.37 of software. When I use 25 FPS with 25 Boxing (default for 1920x1080 from drop down), the playback is speeded up. When I use 24 FPS with 24 Boxing, The playback is speeded up by 2 sec in a 45 sec video What setting do I have to use to get exact playback that matches the stop watch side by side? I think there is a bug there. When I select the 1920x1080 preset, it fills the fields with 25 fps and 25 for boxing which is not ok. Thanks.

[roberttidey]

The video fps is what controls what frame rate the camera is set to do. The boxing fps is how the conversion to mp4 is instructed to label the playback rate of converted file. It does not change the number of frames.

Normally these are set to the same value when selecting most of the presets which should mean it plays back in real time. So I don't understand your comment about the 1920x1080 preset.

By recording at one rate and boxing at another one can speed up or slow down the speed of the playback. Commonly this would be used to say record at 5 fps and box at 25 fps to get 5x speed up on playback.

I did a test at 1920 25/25. A 60 second recording played back in 60 seconds for me. I'm not sure at the moment what could be causing a speed up on your system.

Note that the duration shown in the thumbnail icons is just an indicator and is not accurate as it based on file time stamps. You can accurately get the duration of a recording by looking at the annotation times in the first and last frames.

I would need more information to check further; like the uconfig in use when a recording is made.

[PicoPi]

Thanks for the reply. By preset, I mean selecting "Full HD 1080p 16:9" in camera setting under resolution. It automatically sets video fps and boxing to 25. This produces videos that is faster than life. I recorded a video of a timer on my phone, and played it back side by side with the actual phone time beside the video on the monitor. Its considerably faster. I don't what uconfig is. If you tell me, I can send it to you. I can also make a small video and show you if you like. Since the Raspberry pi camera can do 1080p @ 30fps, I'd like to be able to record and playback at that resolution and speed. As it stands, the playback is not at correct speed. Thanks

[roberttidey]

It is supposed to default to 25/25 for that preset. That produces recording and playback at 25 fps and on my system this plays at the normal speed. My 60 second recording took exactly 60 seconds to playback as measured by a stopwatch.

uconfig is a file in the web install folder (e.g. /var/www/html) which contains a users configuration settings that are different to the default set up. By getting this I can recreate the exact same set up. The log file (under schedule settings) is also useful as it shows when recordings start and stop.

If you can make a sample video available that would be helpful.

I would also suggest downloading the mp4 file and playing that back on other platforms to check if you still see speed up.

In principle it should do 30fps but it is possible there could be speed issues with that. I normally use 25 fps but will do some tests at 30fps

[roberttidey]

There does seem to be an issue with fps > 25. Although the software is requesting 30fps capture the camera is still returning 25 fps. So a recording with 30/30 will run fast. 25/25 does run fine on my set up.

I am checking further but it might be dependent on the camera module used. My test was done on a v1 compatible unit and it could be associated with that. What type of camera module are you using?

[PicoPi]

Thanks for the reply. The uconfig file has only these lines in it; annotation width 1024 quality 100 motion_external 0 I am using V2 of Raspberry camera with a PI3. I have tried watching the mp4 file on my phone and it does the same thing. I've always downloaded the mp4 and watch it on my windows machine. That is the reason I am using your software. It's to be able to control the camera over-head and record my lab bench from VNC on my windows machine. Its obvious that there are differences between our experiences. Here is a link to the mp4 file. If you use your own stop watch, you can see it's running fast. This is at 25/25 default. 30/30 is even faster. I hope you can fix it. I like your software and its exactly what I need. Thanks https://drive.google.com/open?id=1VNAlL6b55XlbJdi1qF5C0-m_01uDzWm4

Thanks.

[PicoPi]

Also, I am not using the motion part. Using the software to control the camera (start, stop,...) Thanks

[roberttidey]

The video shows the playback rate is set to 24 fps which would explain some of the speed up but not all.

The playback rate is normally set by the MP4Box operation and the fps should be set by the boxing value in camera settings.

The actual command used is set by the MP4Box_cmd value in /etc/raspimlpeg like

MP4Box_cmd (set -e;MP4Box -fps %i -add %s %s > /dev/null 2>&1;rm "%s";) &

The %i value is substituted by the camera settings boxing value.

Can you check your /etc/raspimjpeg to see if it has that setting?

But it also looks like the original recording rate is higher than 25 fps, nearer to 30 fps.

Have you changed settings in the /etc/raspimjpeg file? If so then please post it here (zipped)

[PicoPi]

I haven't changed anything since install. Didn't know about these files. I've recorded a new one set at 25/25. As you can see, its even faster. I've also included the raspimgjpg.txt which I copied and pasted to notepad. Let me know if you need anything else. Thanks. https://drive.google.com/file/d/1mKQNjuXvxlmxsqtk4PrhU1j2dVK0-6v2/view?usp=sharing https://drive.google.com/file/d/1WMBe7IEf6N5wGjScFE5HpdjhzZpLP1DA/view?usp=sharing

[roberttidey]

Thanks. The second one does show a playback rate of 25 fps

What would also be good is to get a recording with annotation set to %h:%m:%s %k

This gives me another time reference including the %k which is the frame count in each second.

My current theory is that occasionally a frame is being skipped during the recording. I'm checking that out more carefully. I don't think this a specific problem with the software as it is working fine in a lot of systems without frame loss.

It might be worth checking the ribbon connections to the camera carefully at both ends, and push down the little ribbon cable on the camera module itself. A bad connection could cause frames to be missed. Likewise make sure the power supply and cable to the Pi is good. The camera takes a bit more current and small dips could also cause frame loss. If you are using a USB cable to power it then it needs to be low resistance. Many USB cables can have resistances over 0.5 ohm and that will cause problems.

[PicoPi]

I re-connected the camera ribbon cable from both sides, change the power supply to a real raspberry pi 3A power supply and included the time stamp %h:%m:%s %k No change was noticed from before. Here is the upload Let me know if you need me to do anything else. Thanks. https://drive.google.com/open?id=1Yr2Sb9KF9CH6tRq9mNibTlxMqWZMXh_D

[PicoPi]

I played the last file using potplayer and I went through frames (with F) and it has 15 fps! consistently! There is the problem. It's recording 15 fps for some reason. It doesn't drop a single frame.

[PicoPi]

disregard last message. I am full of shit

[roberttidey]

I have checked out your latest video recording with the frame count. I put it into a video editor which allows me to step through it frame by frame. There is definitely something going wrong with the recording process on your system.

The frame count gets reset to 0 when the second ticks over and then should increment on each frame of the recorded video (0-24). This is what happens correctly on all my cameras and I have not heard reports from other users of a similar problem.

On yours this is not happening with some frames being skipped. E.g. 0,2,2,3,4,5,6,7,8,9,10,11,13,15,15,0 This means only 15 frames got recorded in this 1 second instead of 25.

Do you have another camera module to try out to see if it is associated with that module?

[PicoPi]

I searched for and found a NOIR v1.0 and replaced it with this one. Same problem! Here is a list of programs installed on this pi3. I don't know what else I could take out that won't make the system unstable. Its hard to start from scratch because this pi boots and operates on a fast usb flash dongle and has no SD card. It's about twice as fast as other pi3 i have which has a 12MB/S write. This drive has a 15MB/S write and 30MB/s read. progs.txt Let me know if I can do anything else. At least we know the problem is not the camera.

[roberttidey]

I'll check out the program list.

If you have a spare SD card around it would be good to put just raspbian and the web camera software on. This would help establish if it is something in the software environment causing the problem.

[PicoPi]

As further test, I issued these commands from the command line. raspivid -o myvid.h264 -t 60000 MP4Box -fps 30 -add myvid.h264 myvid.mp4 //to convert to mp4 played perfect on my pc. The problem is your software for sure. I hope you have enough info to hunt it down. Thanks

[roberttidey]

OK. I am still looking.

The problem is that this doesn't reproduce on other systems. It is possible that there is some dependency in your environment that is affecting the raspimjeg capture but not raspivid. That is why it would be good to have a plain SD test so that I can concentrate on any differences.

[PicoPi]

Ok. I understand. I have another RPI3 which has a defective WiFi, and uses an external USB wifi dongle. It has a 16GB SD card. I will back up the SD card and reinstall Raspbian Stretch Lite on it and check. I can't use the SD card on the current RPI3 since the mod to use USB flash drive as a boot drive, is not reversible. I will let you know in a few hours.

[PicoPi]

I installed a fresh copy of OS and Rpi Web software and Murphy fought me tooth and nails to get up and running. It works fine now and the video is normal speed. I haven't analyzed to see if it drops any frames. Its good enough as is and I don't have time to mess with it more. I think you'll have this problem again with another user if you don't figure out what went wrong. I didn't do anything special with the other pi just installed mosquitto and node-red and uninstalled a few stuff like bluej and scratch and a few other programs I'll never use. I never had any problem with my RPI2 B+ and this software. I'll have to find a way to get the flash drive working with a fresh install and start over. However, I am not sure if this software would be what I would use, now that it acted up like this! Imagine if I had complicated stuff going on? I'll never figure out what causes this abnormal operation. If it was a windows OS, I would figure it out in no time. I am a noob in linux and can't be peeking and poking like I can on windows. If there was dependency issues, I don't know what they were that would cause a program to "Kinda work"! Thanks for your help.

[PicoPi]

Found the problem! If you change the preview quality of 10 default to say 100 OR change the width from 512 to say 1024, you'll get exactly the problem I mentioned. I changed it from default to have a bigger preview. That is where the problem lies. Now you have something to go by. Good luck

[roberttidey]

OK. Thanks for the further testing. That may help explain what is going on and would be consistent with raspivid working OK as there is no network preview going on.

First changing those preview settings do not cause a problem on my systems but I am using wired LAN network connections to all my cameras.I use power over ethernet to keep cabling simple and get a solid network connection.

If you are using wifi to a remote browser then it is possible that if you increase the preview bandwidth by increasing the size and the quality then you are introducing latency into the system and that in turn is delaying the response to the video streaming call backs and causing a frame loss.

Note that the quality parameter is very non linear ( a function of the GU compression) and going above 25 will have only a marginal effect on quality.

If you wanted to check this further then you could try temporarily using a wired network connection.

If the wifi bandwidth / latency is the issue and you want the large preview size then I would not increase the quality setting, and consider dropping the preview refresh rate a bit.

[PicoPi]

That's exactly what I reasoned and that's how I found the culprit. I use WiFi only. I can live with the defaults. If it doesn't do that for you, its clearly a lower BW of Wifi. I have no use for Ethernet in my setup. Thanks.

[roberttidey]

There is one setting that might help to step up preview settings in your environment.

In /etc/raspimjpeg there is a setting h264_buffers 0

This controls the number of video stream buffers used. The 0 means use the internal camera default which is 1

If you increase this a bit to say 4 then that could potentially help.

After editing the raspimjpeg config the camera needs to be stopped and started to take effect.

[PicoPi]

Thanks. I'll give that a try tomorrow. I am bushed for today. I'll let you know how much I can push it till it craps out.

[PicoPi]

It got worse! I changed it back to 0. Now I noticed more problems. 1- It stops recording all by its own after around 30 sec. It seems to be arbitrary. Even the button changes state from orange. 2- It doesn't drop frames for around 32 sec, then it drops over 6 sec or recording. All this with the smallest preview possible (128) and quality (5) https://drive.google.com/open?id=1GJ8zFBbmDUpGHpc4KsARty0Ixa3P8LuG 3- If you change the preview to 720, it crashes! and on and on... Sorry, I don't have time to be your tester and find bugs for you. It may work as you claim using Ethernet, but I can't see people running cables to remote cameras. I will write my own in python using raspivid. Good luck

[roberttidey]

OK.

Note that this has been used by many others over wifi successfully. I have also tested using wifi and had no problems. I personally use wired connections because you need at least one wire for powering and POE does this and gives a more secure connection.

This software has been around a long time now and I am not trying to find testers for the basic functionality.