Recieving garbage data from OV5640 with PIXFORMAT_RAW

[Arsanios]

Hello,

I am trying to acquire a RAW frame from OV5640, but I get garbage data, even with the test pattern set. I implemented the low-level raw setting in ll_cam.c file and it works fine, I tested it with OV2640 and It works perfectly.

I have noticed also that with ov5640 with raw setting the sensor sends 4 times the frame size, if I configure the framesize on FRAMESIZE_240X240, I expect a 240x240 = 57600, but every time I receive 230400 bytes which are 240 x 240 x 4. I have noticed that behavior when the esp printed on the serial port that it received a bigger frame than expected, "FB-OVF" .

I checked the RAW register setting and i compared it with the sensor datasheet and it seems set correctly

I tried to acquire RGB565 and JPEG formats from the same sensor OV5640 and they were fine, but i need a RAW format.

Does anyone have any thoughts?

[me-no-dev]

can you post your changes? How many bytes per pixel is the RAW format?

[Arsanios]

First the extension fo ll_cam_set_sample_mode function:

else if (pix_format == PIXFORMAT_RAW) {
        if (sensor_pid != OV2640_PID && sensor_pid != OV5640_PID) {
            ESP_LOGE(TAG, "RAW format is not supported on this sensor");
            return ESP_ERR_NOT_SUPPORTED;
        }

            cam->in_bytes_per_pixel = 1;
            cam->fb_bytes_per_pixel = 1;
            dma_filter = ll_cam_dma_filter_raw;
            sampling_mode = SM_0A0B_0C0D;

    }

then the receiving handler :

static size_t IRAM_ATTR ll_cam_dma_filter_raw(uint8_t* dst, const uint8_t* src, size_t len)
{
    const dma_elem_t* dma_el = (const dma_elem_t*)src;
    size_t elements = len / sizeof(dma_elem_t);
    size_t end = elements / 4;
    for (size_t i = 0; i < end; ++i) {
        dst[0] = dma_el[0].sample1;
        dst[1] = dma_el[0].sample2;
        dst[2] = dma_el[1].sample1;
        dst[3] = dma_el[1].sample2;

        dst[4] = dma_el[2].sample1;
        dst[5] = dma_el[2].sample2;
        dst[6] = dma_el[3].sample1;
        dst[7] = dma_el[3].sample2;
        dma_el += 4;
        dst += 8;
    }
    return elements * 2;
}

It's the same interface that i have used with OV2640 and it worked perfectly as i have mentioned earlier.

Thanks

[me-no-dev]

From what I read in the datasheet, you are getting 4 bytes per RAW pixel. One R, one B and two G values.

[Arsanios]

From what i know, with the RAW format you receive 1 byte/pixel. That byte has 1 color based on BAYER pattern that is applied on the sensor, then you have to perform the demosaicing process to get the other colors by getting the average of the other pixels around the selected one.

If i get RGB then it is not a RAW data, it is RGB data like RGB565 and RGB888

Can you send me the page and the section where you find this point on datasheet?

Thanks

[me-no-dev]

[Arsanios]

These are the bayer pattern options, the first one means that you will receive the BLUE value for the first pixel , the next one will be the Green and so on for the first row. The next row will be GREEN, RED, GREEN and so on, like the following pattern. It is one color per pixel represented in 1 byte

https://benthamopen.com/contents/figures/TOSIGPJ/TOSIGPJ-6-1_F1.jpg

[me-no-dev]

yes, you are correct :) but no idea then why you are getting 4 times the data.

[Arsanios]

I think i found somthing, it is related to scaling function, when i disable it, it worked but with a certain aspect ratio and dimensions.

Can you help me to understand the relation between the numbers in the ratio table

static const ratio_settings_t ratio_table[] = {
//    maxW, maxH, startX, startY,  endX,    endY, offsetX, offsetY,   totX, totY
    { 2560, 1920,   0,       0,    2623,    1951,    32,    16,       2844, 1968 }, //4x3
    { 2560, 1704,   0,     110,    2623,    1843,    32,    16,       2844, 1752 }, //3x2
    { 2560, 1600,   0,     160,    2623,    1791,    32,    16,       2844, 1648 }, //16x10
    { 2560, 1536,   0,     192,    2623,    1759,    32,    16,       2844, 1584 }, //5x3
    { 2560, 1440,   0,     240,    2623,    1711,    32,    16,       2844, 1488 }, //16x9
    { 2560, 1080,   0,     420,    2623,    1531,    32,    16,       2844, 1128 }, //21x9
    { 2400, 1920,  80,       0,    2543,    1951,    32,    16,       2684, 1968 }, //5x4
    { 1920, 1920, 320,       0,    2543,    1951,    32,    16,       2684, 1968 }, //1x1
    { 1088, 1920, 736,       0,    1887,    1951,    32,    16,       1884, 1968 }  //9x16

};

It is in ov5640_setting.h file

I am trying to understand why in certain ratios the start x or y is not 0 and if the totalX or totalY is a combination of the offset and the start values.

Thanks

[Frudawski]

May I ask on what hardware you got RAW data from the ov2640 sensor? ESP32-S, ESP32-S2 or ESP32-S3? I am trying for quite a while now to get RAW data from the sensor, but failed so far. It seems that I have always missing data. Well, maybe I just understand the demosaicing not correctly. What register settings did you used for the ov2640 and ov5640 raw settings?

I have one or two ov5640 somewhere in a drawer, I would happily test your settings and try to figure out the problem.

[Frudawski]

OK, demosaicing was my problem. I will look into the ov5640 RAW problem in the next days.

My RAW settings for ov2640 in ov2460_settings.h:

static const uint8_t ov2640_settings_raw[][2] = {
    {BANK_SEL, BANK_DSP},
    {0xF8,0x01}, 
    {RESET, RESET_DVP},
    {0x05, 0x00}, 
    {IMAGE_MODE, 0x04},
    {0xD7, 0x03},
    {0xE1, 0x77},
    {REG32, 0x03},
    {R_DVP_SP, 0x02},
    {RESET, 0x00}, 
    {0, 0},
};

[Arsanios]

OK, demosaicing was my problem. I will look into the ov5640 RAW problem in the next days.

My RAW settings for ov2640 in ov2460_settings.h:

static const uint8_t ov2640_settings_raw[][2] = {
    {BANK_SEL, BANK_DSP},
    {0xF8,0x01}, 
    {RESET, RESET_DVP},
    {0x05, 0x00}, 
    {IMAGE_MODE, 0x04},
    {0xD7, 0x03},
    {0xE1, 0x77},
    {REG32, 0x03},
    {R_DVP_SP, 0x02},
    {RESET, 0x00}, 
    {0, 0},
};

My ov2640 raw setting is :

static const uint8_t ov2640_settings_raw[][2] = {
    {BANK_SEL, BANK_DSP},
    {RESET, RESET_DVP},
    {IMAGE_MODE, IMAGE_MODE_RAW10},
    {0xD7, 0x03},
    {0xE1, 0x77},
    {RESET, 0x00},
    {0, 0},
};

This configuration worked with me perfectly, my problem now is to set the ov5640 raw settings correctly and receive a 1920x1080 frame in a correct way

Thanks

[jossvall]

OK, demosaicing was my problem. I will look into the ov5640 RAW problem in the next days. My RAW settings for ov2640 in ov2460_settings.h:

static const uint8_t ov2640_settings_raw[][2] = {
    {BANK_SEL, BANK_DSP},
    {0xF8,0x01}, 
    {RESET, RESET_DVP},
    {0x05, 0x00}, 
    {IMAGE_MODE, 0x04},
    {0xD7, 0x03},
    {0xE1, 0x77},
    {REG32, 0x03},
    {R_DVP_SP, 0x02},
    {RESET, 0x00}, 
    {0, 0},
};

My ov2640 raw setting is :

static const uint8_t ov2640_settings_raw[][2] = {
    {BANK_SEL, BANK_DSP},
    {RESET, RESET_DVP},
    {IMAGE_MODE, IMAGE_MODE_RAW10},
    {0xD7, 0x03},
    {0xE1, 0x77},
    {RESET, 0x00},
    {0, 0},
};

This configuration worked with me perfectly, my problem now is to set the ov5640 raw settings correctly and receive a 1920x1080 frame in a correct way

Thanks

Hello there

I have been trying to improve the quality of the image captured with the camera, the quality difference between grayscale and JPEG image is huge!

So I want to obtain the image in raw (I am using the OV2640 sensor) but I have problems with obtaining an image in this format since the project does not have support for this, there is the possibility that you share the code that works for you to obtain an image in RAW with the OV2640 sensor?

[github-actions[bot]]

This issue appears to be stale. Please close it if its no longer valid.

[lance-mlabs]

Hey @Arsanios many thanks for sharing your config for ov2640_settings_raw! I am working on getting ov2640 RAW10 bit image stream working. Are you able to share the updates you presumably made to convert_line_format in to_jpg to handle the processing of PIXFORMAT_RAW frames ? Were you streaming the raw MJPG or doing on board demosaicing as well?