esp_camera_fb_get() failed

[Herwey]

Hi : I try to use esp_camera_fb_get() to get camera buffer, but it stops. I find the camera driver, it stops here. "xQueueReceive(s_state->fb_out, &fb, portMAX_DELAY);" I think camera has run, because it print "I (812) camera: i2s_run". And the same camera in another esp32 works fine. The api function as below : camera_fb_t esp_camera_fb_get() { if (s_state == NULL) { return NULL; } if(!I2S0.conf.rx_start) { if(s_state->config.fb_count > 1) { ESP_LOGD(TAG, "i2s_run"); } i2s_run(); } if(s_state->config.fb_count == 1) { xSemaphoreTake(s_state->frame_ready, portMAX_DELAY); } if(s_state->config.fb_count == 1) { return (camera_fb_t)s_state->fb; } camera_fb_int_t fb = NULL; if(s_state->fb_out) { printf("s_state->fb_out\r\n" ); xQueueReceive(s_state->fb_out, &fb, portMAX_DELAY); printf("s_state->fb_out complete\r\n" ); } return (camera_fb_t)fb; }

[me-no-dev]

there must be something wrong with the hardware :) the function is fine :) it's written correctly and will work fine if the rest is OK. Did it print something about not being able to get VSync?

[Mnark]

I have hit the same issue and agree the problem lies outside the library code...

However, if you replace the line ... xQueueReceive(s_state->fb_out, &fb, portMAX_DELAY);

with ...

      if (xQueueReceive(s_state->fb_out, &fb, 10 * 1000 / portTICK_PERIOD_MS) == pdFALSE) // Wait 10 seconds

       {

            ESP_LOGE(TAG, "Timeout waiting for camera frame");

            return NULL;

        };

control will return to the calling program which can then react accordingly (e.g. re-initialise the camera.)

[me-no-dev]

what is the real issue though? what are you doing in the code that would cause that? if the camera starts normally it should never get into situation like this

[Mnark]

I have yet to figure out the underlying cause (I can read the PID from the sensor so comms do exist), and these changes will not fix the problem, but it will give users of the module an indication of what the underlying problem is rather than just waiting forever, which is what happens now.

There is a similar potential issue few lines before it (checking the semaphore), which can be caused by calling esp_camera_fb_get(fb); twice without calling esp_camera_fb_return(fb); in between.

The timeout could probably reduced to one second from the 10 in the example I gave.(The sensor should answer within a second?)

I think I'm getting close to the issue, and I'm sure its going to be a schoolboy error in my code when I do.

[me-no-dev]

if you find the reason maybe we can better handle the case in the driver in the future :) that is why I am asking. I could also add the camera thread to WDT, which will trigger if the thread is waiting for more than the WDT period.

[me-no-dev]

BTW please also post the settings that you are using. It could be some edge case, though I have tested a lot all options and resolutions.

[Mnark]

I think I have found at least 3 way to get the program into a 'hung' state....

1. Developer Error: forgot to define the configuration as static!

2. Forget to put the camera into 'run' mode (2460 series).

3. Unsupported card - If the card is from a known series, but a different model the configuration succeeds but with the wrong sensor drivers loaded.

I have/had all 3 !!!

[me-no-dev]

1. Why would this be a problem? Config should be copied to the driver... I'll doublecheck.
2. What do you mean by "run" mode? Is that the PWDN issues we discussed (and address 0x3C)?
3. Please give an example for this :)

[Herwey]

The problem has been solved, however, I didn't find the cause :(
I forgot what I changed , but it worked fine......... The same hardware

[JB-DX]

I have the same problem. Every second a camera shot is taken. After many hours that gets stucked in esp_camera_fb_get(). I added debg msgs, and found it waits for the semaphore if(s_state->config.fb_count == 1) { xSemaphoreTake(s_state->frame_ready, portMAX_DELAY); } Since it gets released internally, me-no-dev should know better what happens, I hope. I have seen the get-stuck-problem at least with 2 modules, so it's not h/w dependend. Also as said earlier, I would be wise to return with NULL after waiting for 1-2 seconds. My task executes other 1-second functions, that are now not being called and brings the entire system to a freeze. These should continue even if the camera has probelms. Of course that does not fix the reason.

I can add more debug msg to track it down further. Unfortunately I have to wait many hours before it hits again.

[Mnark]

@JB-DX I'm still digging here...

Interesting that you hit this condition after several hours of running, all the instances I have come across so far are happening during the initialisation. There appear to be a number of 'features' in the existing code that are contributing to this.

One thought that springs to mind is that the pull-up resistor on the PWDN line (pin 32) is not beng initialised?

Pin 32 is the power line for the camera, if it goes low the camera turns itself off.

The library only configures the pull-up if the PWDN pin is defined (>-1).

If the PWDN pin is not defined, the camera will be on initially, but 'may' turn off unexpectedly.

[JB-DX]

@Mnark, There's no PWDN entry in sdkconfig or make menuconfig. Should I add that? I just discovered another stall, and that looks stranger. I had inserted the dbg-msg (marked *1) to see the value of config.fb_count

This stall (after 15 hours) did not hang at the SemaphoreTake and never got to the printf //*1 The last printf was right before the call to esp_camera_fb_get() (see below) So it seems it got hung in the first few lines. Does it match your assumption with PWDN ?

I will add more printf to narrow it down.

camera_fb_t* esp_camera_fb_get()
{
    if (s_state == NULL) {
        return NULL;
    }
    if(!I2S0.conf.rx_start) {
        if(s_state->config.fb_count > 1) {
            ESP_LOGD(TAG, "i2s_run");
        }
        if (i2s_run() != 0) {
            return NULL;
        }
    }
printf("-- camera_fb_get-a %X\n", s_state->config.fb_count);     //*1
    if(s_state->config.fb_count == 1) {
        xSemaphoreTake(s_state->frame_ready, portMAX_DELAY);
    }
printf("-- camera_fb_get-A\n");    
....
}

caller side:

 printf("before esp_camera_fb_get()");
  fb = esp_camera_fb_get();
 printf("after esp_camera_fb_get %p", fb);

[Mnark]

I would recommend adding a PWDN value of 32 to your set-up, this should resolve any problems with your camera power.

BUT... there is always a but, the logic in the initialisation also 'flicks' the power switch on and off multiple times if this pin is defined (for no good reason as far as I can figure), this might give you issues with the camera not starting properly in the first place.

I have a version of code that will initialise the camera successfully every time, but the rest of it still doesn't work... yet.

[Mnark]

Looking at the code above, the only other place it could stall is i2s_run().

I think I did find a 'feature' in there that would hang it, but haven't totally worked it out yet, so don't take this as gospel...

There is a mismatched between the way that the core libraries decide whether to use queues or semaphores. Neither seem to be doing the 'standard' way of just calling malloc and then identifying what type of memory was allocated.

The core seems to just use the presence of the CONFIG_SPIRAM_USE_MALLOC flag to do this, but just because malloc could assign PSRAM, doesn't mean it will.

[JB-DX]

I added more printf's and set config.pin_pwdn = 32 After 7 hours it stalled again, i2s_run() does not return. So setting pwdn to 32 did not improve anything.

[Mnark]

In that case I suspect a memory leak...

Could you amend your printf statements to display free memory (both DRAM and SPIRAM)?

This would either confirm or dismiss the theory before digging into the code to identify it.

[JB-DX]

I already had the same thought and used xPortGetFreeHeapSize() It reports around 2292944 and does not decrease over the hours, How can I get the values for DRAM & SPIRAM separately?

[JB-DX]

The free heap now as total, internal, and external: Right at start: heap=2292572, in=130424, ex=98912

12479 seconds later (3.46 hours) right before stall: heap=2292320, in=130172, ex=98660

So it basicly has not changed.

[JB-DX]

In a different error case esp_camera_fb_get() returned NULL, because s_state was NULL

    if (s_state == NULL) {
        return NULL;
    }

Then I tried resetting the ESP32 with esp_restart(), the ESP32 restarted, but the camera did not initialize as you can see here:

---camera PWDN-Pin= 32
I (1434) sccb: pin_sda 26 pin_scl 27
I (1444) gpio: GPIO[32]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
E (1484) sccb: SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1
E (1484) sccb: SCCB_Write Failed addr:0x30, reg:0x12, data:0x80, ret:-1
E (1504) camera: Detected camera not supported.
E (1504) camera: Camera probe failed with error 0x20004
E (1504) app_camera: Camera init failed with error 0x20004

I tried several times without success. Only after turning off/on power it would start again. It seems the camera reset circuit had no effect. The schematics shows, the CAM-RST is just derived from 3.3V. IO32 only controls the 2.8V and 1.2V power, not the reset line. So how can the camera get out of a weird state without turning off 3.3V? A recuperation via a software reset seems to be impossible?

[Mnark]

I have come across several cases where a software crash results in a system reboot where the camera then fails to initialise. I haven't dug down into the nitty-gritty yet, but this is probably by design as when you are wakening from a 'sleep' the camera keeps its last settings (I think).

A software reset in these instances is never going to work, as you can't talk to the camera. Theoretically, pull pin 32 high (switch off the camera), wait a few ticks, pull pin 32 low (turn the camera on), wait a few ticks for it to warm up, initialise SCCB (i2c) and start talking to the camera... theoretically

I assume that when you say the s_state is NULL, this is after a reboot? If it's losing its s_state while running, that's a whole different problem :)

[JB-DX]

I am not using sleep mode, because every second a shot is taken.

about s_state = NULL case : The first occurence was after some hours of normal operation, and continued from then on with each 1s call. After that it happens right after each esp_restart()

I see that after each reset pin-32 goes high for about 20ms, but that did not help here. I suspect because the camera still gets 3.3V the i2c interface of the camera does not get reset. Maybe the 20ms is too short? I doubt it. Unfortunately this case only happened once, so is quite rare, but when it occurs there's no way out of it. I suspect if CAM-RST would have been connected to an I/O pin, it would have helped here.

---

With an 8051 system I once had a problem that after a watchdog reset the connected i2c interface of an eeprom was not in the proper initial state to accept a new address byte. The solution was to toggle the scl as long as I saw a response on sda and then start again. (I have to search to find back the exact details) This I could easily provoke by firing high voltage discharges on the 8051-system with an ESD gun

[Mnark]

I have no idea what an 8051 system is... but it all sounds very low-level.

I cloned the Espressif code and stripped out a lot of the code around resetting, if I ever get all the bugs out of it I will share.. The only delays I think are needed are, a very short one if the pin is going high (turning camera off), and a slightly longer one if turning the camera on (let the valves warm up!).

Your real problem though is why is your system losing it's state after 6 hours of running... the camera hanging sounds like a symptom, not a cause.

[JB-DX]

I had a case again where the camera did not get reset even after several esp_restart(), similar to what you noted. Unfortunately this module isn't connected to a terminal to see the debug messages.

For the real problem I found that in esp_camera_fb_get() I2S0.conf.rx_start remains 0, and waits forever in this part of i2s_run()

    // wait for frame
    camera_fb_int_t *fb = s_state->fb;
    while(s_state->config.fb_count > 1) {
        while(s_state->fb->ref && s_state->fb->next != fb) {
            s_state->fb = s_state->fb->next;
        }
        if(s_state->fb->ref == 0) {
            break;
        }
        vTaskDelay(2);
    }

I have not found out the cause, maybe it's indeed just a symptom. I don't know how to get closer to the problem. As a workaround I try now to esp_restart() in this case. Still need to see if that is reliable. Is there a way to check the status of the camera in some way, to see the difference between normal and stall condition?

---

8051 is a indeed a low-level system. Back then when the watchdog-reset hit, the I2C eeprom was sometimes in the middle of a read operation. So sending a start-condition had no effect. At reset I then did this 10 times: Read one more bit, if SDA is low read the next bit, if it is high send a start-condition. After max 10 bits the start-condition was finally recognized and the eeprom responded normally.

[JB-DX]

I had this init error again SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1 and used my ancient i2c-init approach with success. After a esp_restart() no more SCCB errors. tryRecoverI2C() is called right before esp_camera_init() no matter if the error exists or not. It recovered just once, now I need to see if that works permanently, so even longer test periods are required..

void  sendI2cStop(int scl, int sda)
{
 gpio_set_direction(sda, GPIO_MODE_OUTPUT);
 gpio_set_level(sda,0);
 vTaskDelay(1);
 gpio_set_level(scl,1);
 vTaskDelay(1);
 gpio_set_level(sda,1);
 vTaskDelay(1);
 gpio_set_direction(sda, GPIO_MODE_INPUT);
 vTaskDelay(1);
}

void  tryRecoverI2C(void)
{
 int n,i;
#define sda SIOD_GPIO_NUM
#define scl SIOC_GPIO_NUM

 gpio_set_level(scl,0);
 gpio_set_direction(sda, GPIO_MODE_INPUT);
 gpio_set_direction(scl, GPIO_MODE_OUTPUT);
 gpio_set_level(scl,0);
 gpio_set_level(sda,1);
 for (i=0; i<10; i++)
  {
   gpio_set_level(scl,0);
   vTaskDelay(1);
   n= gpio_get_level(sda);
   gpio_set_level(scl,1);
   vTaskDelay(1);
   gpio_set_level(scl,0);
   if (n)
    sendI2cStop(scl, sda);
  }

[git6499]

I had the time-out error with requested RGB and YUV format, but not with grayscale and jpeg.

[JB-DX]

The timeout occurs one to three times per day, during the past 5 months. A watchdog task then triggers a software reset, and that works fine and reliable. This is the current log of the last 4 days (T= day)

0T 14:42:48   
1T 10:55:00   
2T  8:50:10 
3T  7:00:38 

[xenpac]

interesting thread here.... i have a custom board with a LAN8720 and an OV7670 cam on it.

modded the driver so the cam gets inited. esp_camera_init(&camera_config); ..........returns no errors.

So CAM is up, but not running. No I2S-interrupt ever hits. (all signal can be seen with Scope)

"E (9120) camera: Failed to get the frame on time!" is shown.

Looking for a way to verify that pixelclocks are reaching the I2S . Is there any register to check for? Thanks

[me-no-dev]

the error means that VREF signal is not present. That points either to missing XCLK or to bad config which caused the sensor to not output the pixels. logic analyser could help :)

[xenpac]

Thanks, io34 = VSYNC, normal low with short positive pulses io35 = HREF, normally high with short negativ pulses io32 = pixelclock, nice signal at 12.5 mhz there is no xclk because cam has oscillator onboard.(extern hardware oscillator)

• disabled "camera_enable_out_clock()" in driver.

Is there a I2S-register-statusbit that indicates " i am being clocked" ?

[me-no-dev]

I tested 12.5 MHz and it was fine. Signals you say though I am not sure. I am not familiar with the camera sensor so I can not tell you what to change, but there should be registers that control the signals polarity and behaviour. Your VSYNC seems inverted. HREF should be LOW when pixel data is being sent (each line).

[xenpac]

OK, somewhere i red that pixelclock can only be max 20mhz. i that correct?

[me-no-dev]

I would say, stick to 10 if you want good results.

[xenpac]

I am currently struggeling with "bad" -marked framebuffers caused by "xQueueSendFromISR" could not queue a dma-buffer to filtertask. If i disable the bad-marking, frames are retrieved fast (with bad data;-) If i leave bad-marking on,i dont receive anything.

hmmm.....something is not fastenough ?

" // we got one dma buffer filled. called from i2s_isr

static void IRAM_ATTR signal_dma_buf_received(bool need_yield) { size_t dma_desc_filled = s_state->dma_desc_cur; // save current s_state->dma_desc_cur = (dma_desc_filled + 1) % s_state->dma_desc_count; // update next dma bufferpointer to be used by i2s s_state->dma_received_count++; // inc number of dma bufers received Icnt3++; if(!s_state->fb->ref && s_state->fb->bad) // if current framebuffer has bad data, skip dma-buffer from i2s { Icnt4++; need_yield = false; // skip it return; }

// tell dma-filter-task to process this buffer
BaseType_t higher_priority_task_woken;
BaseType_t ret = xQueueSendFromISR(s_state->data_ready_q, &dma_desc_filled, &higher_priority_task_woken); // signal filter task
if (ret != pdTRUE) // if dma-buffer could not be send, the framebuffer gets corrupt, so bad.  Tomk !!! this fails
{

Icnt5++;

    if(!s_state->fb->ref) // only a few are wrong!
    {
        s_state->fb->bad = 1; // mark framebuffer is missing data. tomk. Icnt3-5: disabled:75840 0 11657; enabled: 120261 109778 127
    }
}
*need_yield = (ret == pdTRUE && higher_priority_task_woken == pdTRUE);// =1, if could be queued and the task was woken.

} "

[xenpac]

how do you insert code here???;-)

[7FM]

Like that: ``` code... ```

[xenpac]

hmmm... i tried this <> button(code insert. it creates 2 '' then i inserted the code between these 2 '' but......looks ugly.

do i need to use 3''' and3 ''' ?

Bytheway......

I resolved my issue. The camera pixelclock was too high (12.5 mhz) for 640X480 yuv. The queue from the i2s_isr to dma_filter_task got overflown....too fast.

I lowered pclk to 5 Mhz and now it runs fine. ;-)

Just for the records: ESP/I2S interface specs for the camera: D0-D7 = must be stable on rising edge of pixelclock PCLK = rising edge of pixelclock triggers Data Capture HREF = horizontal reference (HSYNC), must be "high" for Data to be captuered VSYNC = vertical sync must be "high" for Data to be captured.

[7FM]

I think the <> button is made for just one line of code and `` for code blocks, but not sure I only know that \``` works fine. And there might be a difference between ' and

[JB-DX]

Inserting code is easy only when you know it. I also had the same problem in the beginning.

Insert your code as normal text. Then mark that range. Now click on the <> button. Click on Preview to see your result.

[7FM]

Good to know, thanks!

[xenpac]

Ahaaaa.......thanks :-)

[xenpac]

my ov7670 is getting captured. (YUV422) however the colors are wrong.

I (18975) webserver: JPG: 35443B 2451ms

So the webserver transcodes yuv422-input (614400 Bytes) to jpg, resulting in picturesize of 35443Bytes.

My browser displays this:

hmmm......

[me-no-dev]

looks like maybe the order of yuyv is wrong? like the Y channel is instead U and V

[me-no-dev]

how would a grayscale look like? since it has the same settings as yuv (and takes only the Y channel), if you get the grayscale to work correctly, then you can move over to see if the U and V channels are on the correct order.

[xenpac]

Indeed...;)

OV7670 default/reset yuv-format is: UYVY We need: YUYV according reg-settins are: SCCB_Write(sensor->slv_addr, 0x3a, 0x04); //Format: reg0x3a[3], reg0x3d[1]. SCCB_Write(sensor->slv_addr, 0x3d, 0x99); //00 = YUYV

Though colors are still "pale", but colorbar is correct.

[me-no-dev]

the upper image looks like it's a shot in the dark and night-mode is on (looks more or less grayscale)

[me-no-dev]

the black border on the left suggests that the window is not properly configured and an offset on X is missing. Most if not all OV sensor use a few rows and columns to measure light and colors and I don't know what else, in order to provide proper white balance and exposure. I don't know if this is the case here, or the image is just shot at dark, but something worth looking at.

[xenpac]

@me-no-dev true, i did not do any init sequence yet. So its the "raw" inital reset setting of the ov7670, just timing and yuv are now set correctly. The left black-pixel are indeed "black-reference-not iluminated" and should be windowed out. Asto the "grayscale" image, true, it looks non-color, during daylight. Will experiment for correct saturation etc. settings.

[morrisho]

Hi there, I am testing ESP32-CAM board and also got the message. [E][camera.c:1344] esp_camera_fb_get(): Failed to get the frame on time! Camera capture failed

I measured PCLK which shows 5Mhz and MCLK(your XCLK) which shows 20Mhz. Are the clock frequencies correct?

[fazaio]

@morrisho hy morrisso what is your camera label?, i have same issue like you, [E][camera.c:1344] esp_camera_fb_get(): Failed to get the frame on time! Camera capture failed.

my camera label is xrz00d1-v240.

[xenpac]

What camera module is that? (always remember, it can be a hardware fault)