[BUG] - libcamera-still timelapse crashes on Raspbian Lite if preview isn't turned off

[sv87411]

Describe the bug I appreciate you would normally turn off preview in a command line only environment, but if you don't and use libcamera-still to create a time-lapse it appears to consume all available memory and after about 27 frames (in my setup) it crashes with a V4L2 error.

Test environment Raspberry Pi Model 3 B V1.2 Raspberry camera module V1.3

OS - Lite version, no GUI

NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Command - note that preview isn't turned off using -n or --nopreview

libcamera-still -t 900000 --timelapse 2000 --framestart 1 -o test%03d.jpg

The command starts OK and creates files called test001.jpg, test002.jpg etc, but after approximately 27 frames it stops with the following error:

[1:31:08.375717860] [4306] ERROR V4L2 v4l2_videodevice.cpp:1158 /dev/video0[15:cap]: Unable to request 1 buffers: Cannot allocate memory
[1:31:08.375872704] [4306] ERROR RPI raspberrypi.cpp:817 Failed to allocate buffers

Up until the point where it errors there are no other errors, even with additional debugging turned on.

All you can do once it has crashed is hit CTRL-C and stop the process. I have noticed that when generating a time-lapse that free memory decreases over time. This happens quite fast when preview is on (until you get the crash), but still happens very slowly when preview is off. I watched the output of free -h while running the time-lapse using:

watch -n 1 free -h

It doesn't appear to fail with preview turned off, but as I say, it does still seem to consume memory, I haven't yet left it for a sufficiently long time with preview turned off to see if it still fails. Not sure why memory goes down if it is simply capturing an image, writing it to disk and tracking some counters during a time-lapse unless there's some bug in V4L2.

Bug report camera-bug-report tool not available on my OS

Additional context I'm hoping to use Node-RED to actually control the time-lapse and will just issue a command at regular intervals to capture a single image using libcamera-still, but I thought it would be useful to raise this as I'd noticed it and it could suggest a memory issue somewhere.

[fire-eggs]

Testing on a RPi4B, there appears to be a memory leak issue.

Running your command line and watching the memory consumption, I see the "used memory" gradually increase. When the app finishes, the "used memory" dropped by over 500 megabytes! That amount of memory would surely overload a RPi3B.

[davidplowman]

Hi, thanks for reporting this. I've tried this on a freshly downloaded Bulsseye and I see the same problem. In fact, I think it may be CMA memory that is running out (try watch -n 1 tail -n 1 /proc/meminfo).

Quite a lot of development has gone on in the meantime, and the latest code seems to have fixed this. We're going to be trying to roll an OS update shortly so hopefully that will fix this. (You could of course build from source in the meantime, but that's quite a lot more effort, and currently involves a kernel update too.)

[sv87411]

Phew, at least it's not just me going mad or having this issue in isolation.

I did some additional testing watching the /proc/meminfo stats and also graphed them to see what was going on and when preview is left on during the time-lapse the CMA drops from around 250MB (on my RPi 3B) very quickly until it reaches about 28MB when it crashes after 56 seconds (around 28 time-lapse frames).

Interestingly when I graphed the CMA memory with the same time-lapse command but with no preview, it started again around 250MB and held steady for about 420 seconds (approx 210 time-lapse frames) and then started to drop (albeit much slower) until it reached about 114MB when my time-lapse command ended after 15 mins (440 time-lapse frames, which is 10 frames fewer than I would have expected a 900 second time-lapse with a 2 second gap to record, but I did noticed some pauses when frames were writing to disk). Therefore, I think it would eventually fail and run out of CMA even with preview turned off.

This is all academic though if the problem is fixed which is good news! 👍️

I don't think I'll be compiling from source and I'm happy to wait until there's an OS updated version of libcamera. Especially as I said I'll probably be controlling my time-lapse from node-RED issuing a remote exec of libcamera-still to capture a single frame per period.

Thank you for the swift response and tip about looking at meminfo/CMA!

[naushir]

Closing this issue, as the latest code has the leak resolved.