Camera image capture example returns images out of sequence

[easytarget]

Board

ESP32 DEV modules with cameras

Device Description

AI-THINKER ESP32-CAM

• but will also affect any board using the esp-cam code/examples

Hardware Configuration

ftdi serial for programming and debug

Version

v2.0.1

IDE Name

Arduino IDE 1.8.18

Operating System

Any

Flash frequency

not relevant

PSRAM enabled

yes

Upload speed

any

Description

Background

As per #5805 Images captured by the CameraWebServer example are 'delayed' by several frames.

This is trivial to reproduce by configuring and running the example, connecting from a browser, stopping the stream and then repeatedly hitting 'Get Still' while moving the camera's pov (or waving a hand in front, etc). The frames returned are always 1 or more images behind the actual camera view when the capture was taken.

This seem to be an effect of the camera working like a queue; with the framebuffer holding (fb_count+1) images.

• When you capture an image with esp_camera_fb_get() it is added to the the framebuffer, pushing out older images as necessary.
• But the returned pointer from this function will be to the oldest image in the framebuffer, not the latest one.
• This can get quite extreme; it is quite possible to have an image that is minutes, hours (or more) old being returned as the 'latest' if you only infrequently trigger a capture, and don't stream at the same time.

Enhancement Request:

The solution (as noted previously) is to insert: config.grab_mode = CAMERA_GRAB_LATEST; ..as line 54 in the examples here and elsewhere.

Or at least document it more loudly; I've lost a whole morning debugging this ;-) It's quite obscure and until I discovered config.grab_mode I had no clue what was happening, and google wasn't being much help.

Possible bug:

NB: I say the framebuffer is fb_count+1 images long, because that is how it appears to work in practice..

• If I set config.fb_count = 2 the returned image lags the capture by three.
• if I force it to 1(default = 2 for my board) I still have a single frame lag.
• If I set it to 6, I see 7 frames of lag.
• etc.
• If set to zero, the sketch fails with 'allocation failed' errors, as expected.

This look suspiciously like there is an underlying 'off by one' error somewhere in the closed-source side of the camera code.

Sketch

This happens with the built-in example.
Files->Examples->ESP32 Dev Module->ESP32->Camera->CameraWebServer

Debug Message

None applicable here

Other Steps to Reproduce

None

I have checked existing issues, online documentation and the Troubleshooting Guide

• [X] I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[VojtechBartoska]

Hello, can you please retest this on v2.0.3-rc1?

[VojtechBartoska]

There were some changes on CameraWebServer, can you please retest your code?

[easytarget]

Bit tricky since I adapted my code to avoid this problem, but I'll see about checking out and running an older version, then testing it again asap.

[easytarget]

Ok, the ESP32 -> Camera example has been fixed

• it now uses config.grab_mode = CAMERA_GRAB_LATEST when appropriate, so that is OK.

But if I use the framebuffer by editing camera config to force:

    config.fb_count       = 1;
    config.grab_mode      = CAMERA_GRAB_WHEN_EMPTY;

There is still an 'off by one' error, the actual number of frames is fb_count + 1

When I use the above configuration and take still images while changing the camera position before each image the framebuffer appears to actually have two entries, not one.

• This should probably be documented somewhere, since it is a bit confusing until you realise that the framebuffer count starts at 0

edit: Tested using the 2.0.3 release.

[me-no-dev]

frame buffer count does not start at 0. That would be very misleading if it was true. when 1 frame buffer is used, the driver can not continuously acquire frames. When you request one, it will go and wait for VSYNC, then grab the frame and give it to you, then stop the bus.

[VojtechBartoska]

@easytarget Can I consider this covered?

[easytarget]

Oh, sorry. Yes this can be closed. The documentation (in headers) for these options is improved and clearer about operation with the framebuffer. And the examples now use 'GRAB_LATEST' when capturing stills, so you get the current image directly.

[wayinone]

Thanks, with the information above, I solved with

    config.fb_count       = 2; // If set to 1, it will have issue here described.
    config.grab_mode      = CAMERA_GRAB_WHEN_EMPTY;