Corrupt JPEG at high quality with high frequency scene

[BrianPugh]

I'm currently using an esp32-cam module which has an ov2640 module and 4096KB of OnBoard RAM. I'm running the CPU at 240MHz and am also running a webserver. I'm currently taking photos at QXGA resolution in JPEG mode. I have an endpoint where a GET command will trigger a photo and send back the jpeg. Usually this works well, but I'm currently having a few issues.

If I have the jpeg quality high (say around 4 or 6), things usually work. However, if I take a picture of a complex scene (say, taking a picture out of my screen door, so a lot of high frequency data, lots of contrast, lots of various colors, basically a lot of data that cannot be compressed very well), then the jpeg becomes corrupt. However, pictures of "simpler scenes" come through fine. Taking the same picture with a lower jpeg quality (10) results in a complete image that nearly fills the allocated buffer (297KB image in a 375KB frame buffer). After doing a little bit of debugging, I found that the corrupt images don't have the jpeg EOI marker, so something is getting corrupt. Sometimes the corrupt jpeg is just the first few lines of the image, sometimes it's upward of 30% of the lines.

Any ideas on what this may be or how to debug it further? I suspect it has something to do with the OV2640 sending barely compressed data due to the complexity of the scene.

[Schaggo]

I experimented a lot to reach an equilibrium of max image size in bytes and max speed to get a frame in stable operation for still images. I encountered your problem and think it is caused by two things. Nr.1 seems to be that the PSRAM seems to corrupt anything that reaches over the halfway point of its total size. NR.2 apparently the ESP isn't fast enough to pick up the frame from the cam in some cases.

Some tips: 1) Try to use PSRAM only for the frame buffer and limit the PRSRAM size to 2MB. -> If your image buffer is over 2MB you get incomplete images as you described.

2) When using the camera server example from Espressif, deactivate the face recognition. -> Program size shrinks drastically.

3) Try the following settings when initializing the camera: camera_config.frame_size = FRAMESIZE_UXGA; camera_config.xclk_freq_hz = 16500000; camera_config.jpeg_quality = 1; camera_config.fb_count = 2; -> The xclk_frequenzy is the result of a lot of trial and error. I cannot explain why it works, but it does. 4) I increased the Flash Speed to 80MHz, but after a lot of iterations am not sure if it has any effect.

Sending the raw frame buffer (jpeg_quality = 1) as .jpeg results in uncorrupted images up to 950kb. I was able to repeat this about 70 times and declared victory.

I just streamed for a while here's the serial output: MJPG: 635867B 1145ms (0.9fps), AVG: 1149ms (0.9fps), 0+0+0+0=0 0 MJPG: 635591B 1323ms (0.8fps), AVG: 1155ms (0.9fps), 0+0+0+0=0 0 MJPG: 636130B 1410ms (0.7fps), AVG: 1172ms (0.9fps), 0+0+0+0=0 0 MJPG: 635737B 1305ms (0.8fps), AVG: 1180ms (0.8fps), 0+0+0+0=0 0 MJPG: 635635B 1356ms (0.7fps), AVG: 1197ms (0.8fps), 0+0+0+0=0 0 MJPG: 635287B 1282ms (0.8fps), AVG: 1205ms (0.8fps), 0+0+0+0=0 0 MJPG: 635643B 1379ms (0.7fps), AVG: 1219ms (0.8fps), 0+0+0+0=0 0 MJPG: 635621B 1475ms (0.7fps), AVG: 1242ms (0.8fps), 0+0+0+0=0 0 MJPG: 636696B 1142ms (0.9fps), AVG: 1247ms (0.8fps), 0+0+0+0=0 0 MJPG: 635881B 1257ms (0.8fps), AVG: 1255ms (0.8fps), 0+0+0+0=0 0 MJPG: 636021B 1492ms (0.7fps), AVG: 1274ms (0.8fps), 0+0+0+0=0 0 MJPG: 636000B 1601ms (0.6fps), AVG: 1298ms (0.8fps), 0+0+0+0=0 0 MJPG: 635690B 1462ms (0.7fps), AVG: 1316ms (0.8fps), 0+0+0+0=0 0 MJPG: 636457B 1396ms (0.7fps), AVG: 1337ms (0.7fps), 0+0+0+0=0 0 MJPG: 635866B 1224ms (0.8fps), AVG: 1335ms (0.7fps), 0+0+0+0=0 0 MJPG: 636663B 1101ms (0.9fps), AVG: 1332ms (0.8fps), 0+0+0+0=0 0 MJPG: 635811B 1392ms (0.7fps), AVG: 1338ms (0.7fps), 0+0+0+0=0 0 MJPG: 635797B 1469ms (0.7fps), AVG: 1350ms (0.7fps), 0+0+0+0=0 0 MJPG: 636012B 1287ms (0.8fps), AVG: 1341ms (0.7fps), 0+0+0+0=0 0 MJPG: 635715B 1514ms (0.7fps), AVG: 1350ms (0.7fps), 0+0+0+0=0 0

[BrianPugh]

Hey @Schaggo , thanks for the feedback!

I tried setting those configurations in my application, but it didn't improve.

Here's a video of the issue when using the cam-webserver example (the quality is a little rough due to gif compression, but it gets the idea and the scene across). Make sure to hit the gear to HD instead of SD

https://gfycat.com/pinkwhimsicalaustraliansilkyterrier

Here's a still of the stream working at a lower quality:

[Schaggo]

Your screen door might be as close to the worst case scenario as it gets. Hmm did you limit the PSRAM size? heap_caps_malloc(1024*1024*2, MALLOC_CAP_SPIRAM);

Which version of the driver are you using ? Are you on Arduino or IDF?

[Schaggo]

I was able to reproduce your situation with an image of noise. I think what is happening is that at camera_init the frame buffer size is set to s_state->fb_size = (s_state->width * s_state->height * s_state->fb_bytes_per_pixel) / compression_ratio_bound; if the factual size of the image is larger than the buffer, the problem occurs.

I am currently trying to change compression_ratio_bound to a lower value, but am struggling to access the driver while on Arduino. The driver is in a precompiled archive - for whatever reason - and has to be recompiled.

[BrianPugh]

Thanks for the additional info! I've also been playing with this, but haven't come up with any success. Glad that another person is helping with this issue!

[fiskelin]

I have enter similar problem. I got the corrupted pic using ov2640 even I set camera_config.xclk_freq_hz to 10000000 and increase the frame buffer size

[raduprv]

Me too. It seems that the maximum file size that is saved is around 350 kb (quality 4 or 5). One would think that setting only one frame buffer would help, but now. Btw, how did you increase the frame buffer size?

[raduprv]

Ok, so I fixed the problem with corrupted images at higher quality (not being able to go over ~350kb. I spent days and days playing with the registries, and finally I found the right one and the right value.

s->set_reg(s,0xff,0xff,0x00);//banksel s->set_reg(s,0xd3,0xff,5);//clock

Then wait a bit before reading the frame buffer. Let me know if it works for you.

[BrianPugh]

@raduprv I really hope that works! I'll try it out later today and let you know if it helped!

[raduprv]

If it still doesn't work, use my settings from here (last post). https://github.com/espressif/esp32-camera/issues/203

[jameszah]

Hi -- I have worked on this topic as well.

Here is my post on finding those corrupt jpegs, and getting rid of them: https://github.com/espressif/esp32-camera/issues/162#issuecomment-668222242

And when that proved unsatisfactory, there is my post on setting up the camera so I would never get bad jpegs https://github.com/espressif/esp32-camera/issues/185#issue-716800775

That ~350kb (384kb) limit can be bumped up for 960kb, and I've never had a bad frame since. The only cost is it really eats up your psram, so you can only allocate 3-4 buffers. It is used in the https://github.com/jameszah/ESP32-CAM-Video-Recorder-junior.

It seems I have not updated it recently ... but here are the critical lines:

  config.frame_size = FRAMESIZE_UXGA; // edit in framesizes below

  //v11  -- change qual to 5, meaning fb_count had to go to 4, or 3 with internet framebuffer
  config.jpeg_quality = 5;  // 1 to 63 - smaller number is higher quality and more data - must be lower than the quality parameter at the top
  config.fb_count = 3;

[raduprv]

I read your solutions (actually I read all I could find online about the corrupt JPG problem), but none worked properly. My solution works perfectly, at least for two people. The problem was with the ISP's JPG clock, it was unable to do all the work at the default settings. Changing that clock fixed it, so no more hacks are needed :)

Btw, my problem was never the buffer size limit, I increased it to the max and it always gave me bad results, sometimes even at less than 250kbs of image.

[BrianPugh]

In my tests so far, with the highest quality jpegs, sometimes they hit around 700kb for complex scenes, so the buffer was definitely an issue for me (resolved by hardcoding the compression ratio bound to 2; 1 is a bit excessive. Could probably get by with 3).

[raduprv]

From my tests, a quality rating of 2 is ideal. Better quality is just marginally noticeable, but increases the file size by quite a bit.

[github-actions[bot]]

This issue appears to be stale. Please close it if its no longer valid.