Timelapse fails when using RPiHQ

[cplonski]

ffmpeg crashes when building the timelapse at end of night.

[error]: malloc of size 38600544 failed Aug 29 07:52:37 allsky allsky.sh[14792]: Video encoding failed Aug 29 07:53:31 allsky allsky.sh[14792]: ./timelapse.sh: line 35: 28012 Killed
ffmpeg -y -f image2 -r $FPS -i images/$1/sequence/%04d.$EXTENSION -vcodec libx264 -b:v 2000k -pix_fmt yuv420p -movflags +faststart images/$1/allsky-$1.mp4

[thomasjacquin]

Thanks for the issue report. The images produced by the RPi camera are huge so the RPi runs out of memory while creating the timelapse. I’ll add code to shrink the images prior to running ffmpeg.

[cplonski]

I just tried adding -filter:v scale=1352:1014 to the existing ffmpeg parameters, and it seems to work fine. I chose those values to maintain aspect ratio, yet keeping height below 1080, I divided the original still resolution of 4056x3040 by 3

[across-suffolk]

Thanks @cplonski. I was having the same issue and have put your addendum into timelapse.sh. Sorry if this is a naive question; does the order make a difference or is this correct?

ffmpeg -y -f image2 \ -r $FPS \ -i images/$1/sequence/%04d.$EXTENSION \ -vcodec libx264 \ -filter:v scale=1352:1014 \ -b:v 2000k \ -pix_fmt yuv420p \ -movflags +faststart \ images/$1/allsky-$1.mp4

[cplonski]

Here is what I have, but the order may not be important:

ffmpeg -y -f image2 \ -r $FPS \ -i images/$1/sequence/%04d.$EXTENSION \ -filter:v scale=1352:1014 \ -vcodec libx264 \ -b:v 2000k \ -pix_fmt yuv420p \ -movflags +faststart \ images/$1/allsky-$1.mp4

[across-suffolk]

I have tried adjusting the order of the ffmpeg command settings but this is still not working. I have a RPi 3A+ with a 32GB card and ran the command with the following results, having produced a 12mb file that is unreadable: ./scripts/timelapse.sh: line 36: 12754 Killed

Is this due to the RPi running out of memory? How do I check this? If so, would swapping for a RPi 4 with 4GB ram help?

Edit: I just looked through the allsky.log and found many instances of lsusb: option requires an argument -- 'D'

I can send a zipped copy of the log if that helps.

[WirthmU]

After I have applied the filter settings I get the timelapse is working perfectly well on my RPI4.

I have noticed that there is a error in the cpu information: [libx264 @ 0x1c0bb70] using cpu capabilities: ARMv6 NEON

The cpu in the RPI4 is an ARMv7 processor (ARMv7 Processor rev 3 (v7l)). Can this be configured somewhere in allsky?

[GinoRobert73]

I have the same problem. Even if i used: -filter:v scale=1352:1014. Any idea how to fix it?

[across-suffolk]

My solution was to swap out the RPi3 for a model 4, with 4GB ram. Although the cpu and memory usage still increase a lot during the time-lapse routine, it no longer crashes out and the movie file is created.

[GinoRobert73]

Thanks for the answer. First, i will try to increase the swap file to 2048 and do a test. If it's not working, maybe the solution is to replace the PI board with model 4...

[GinoRobert73]

It's working, timelapse was succesfully generated.

[kevinpcross]

Thanks for the issue report. The images produced by the RPi camera are huge so the RPi runs out of memory while creating the timelapse. I’ll add code to shrink the images prior to running ffmpeg.

Thomas: I created a script that does this exact thing. It tries to keep file sizes used by ffmpeg between 1-2.5 MB. It calculates the average file sizes from all the original files and average dimensions of width and height, then mogrify's them proportionally in another directory. It also insures that the width and height have even dimensional units sense ffmpeg does like odd sizes.

I'm willing to share it with you if you want. Just let me know where you want me to place it.

I'm building a system I call SkyPeek which is basically your Allsky system modified to accommodate usage as a portable observation video record attached to a telescope. Eventually I hope to allow it to give the user an integrated logging ability.

I've got about half of it built and it produces timelapse mpegs on request from a web interface since the purpose of SkyPeek is to record an observation session down the length of the scope tube. The user has the ability then to produce a timelapse right away. The unit weighs in at about 12 Oz and runs on battery and attaches with velcro or magnets to the scope.

Kevin

[kevinpcross]

Hello ALL I haven't heard back from Thomas so I thought I'd drop this file here for others to use.

The attached file is a drop-in replacement for ~allsky/scripts/timelapse.sh. This is the standard install location. Just rename the file timelapse.sh and give it proper execution rights. Save your old file with .orig suffix or something like that. Also make sure you have plenty of free mem space left, ~2.5 - 3.0 times the size of date folder where the mp4 is created is needed

It has resolved the issue of malloc errors from ffmpeg that breaks the creation of the mp4 file. It does take a while to run, ~2-4 sec/image. Your original files are preserved and files are resized automatically. See my previous post for more info.

This file is unsupported, but it shouldn't take much to make it work if that's needed at all. I'd be interested to see if this helps anyone so leave feedback please.

timelapse.sh.txt

[thomasjacquin]

Thanks Kevin, sorry for the late reply, I'm a little back up in my emails. I'll take a look and compare the performance between the scale option and the mogrify one. Thanks for sharing the code.

[robertpascale]

I just tried adding -filter:v scale=1352:1014 to the existing ffmpeg parameters, and it seems to work fine. I chose those values to maintain aspect ratio, yet keeping height below 1080, I divided the original still resolution of 4056x3040 by 3

Confirming this works well and maintains a decent file size.

[svego]

Any progress on this issue? What's the best alternative? Using @kevinpcross alternative file or the filter option?

[kevinpcross]

@svego and all:

My script did break with the same problems as the original when I had some larger image files. If that happens to you you can comment out lines 76-78 in the attached timelapse.sh.txt script earlier in this discussion thread. Those lines are listed below

R_HALFAGAIN=$((REDUCTION/2)) REDUCTION=$((REDUCTION*2)) REDUCTION=$((REDUCTION + R_HALFAGAIN))

This is where I try to regain some resolution instead of strict scaling.

[svego]

I've made the scale option configurable by configs. That way you don't have to edit the scripts and will be able to update with git more easy. Now waiting for @thomasjacquin to accept the new code ;-)

[thomasjacquin]

I have merged @svego's code into the master branch. Thanks for the PR.

[svego]

If you use the latest version of the code, you can use the following setting for the pi camera: TIMELAPSEWIDTH=1442 TIMELAPSEHEIGHT=1080

This works on my pi3.

[EricClaeys]

cplonski, if this problem is resolved, please close the issue.

FYI, I just bought a Pi 4 with 8 GB memory and have 500 MB swap, and timelapse.sh fails for me with an ASI290 (1936x1096 chip), but on my Pi 4 with 4 GB memory and default swap size, it works with an ASI178MC (4096 x 2080). Unfortunately I can't get on the 4 GB Pi to see how it's different. Eric

[EricClaeys]

across-suffolk, FYI, the lsusb: option requires an argument -- 'D'error in the log file is fixed in version 0.8. Eric