Some issues about DMA and I2S

liubenyuan commented 7 years ago

Hi, I found that there is barely any document about the I2S on esp32 (its DMA operations and registers).

In camera.c

why buf_size (and also the size and length of s_dma_desc) is initialized as line_width * 4? In ov7725, if using RGB565 format, the maximum bytes per line is only 2 * line_width. In mt9v034, only monochrome mode is supported, so bytes per line is just line_width.
about SET_PERI_REG_BITS(I2S_RXEOF_NUM_REG(0), I2S_RX_EOF_NUM, (buf_line_width - 2)*2, I2S_RX_EOF_NUM_S); What is the meaning of magic -2 and *2?

In line_filter_task, the following codes confused me

    if (line_count & 1) {
        uint8_t* psrc = s_fb + (line_count - 1) * buf_line_width;
        memcpy(pfb, psrc, buf_line_width);
    }
    else {
        const uint32_t* buf = s_dma_buf[buf_idx];
        for (int i = 0; i < buf_line_width; ++i) {
            uint32_t v = *buf;
            uint8_t comp = (v & 0xff0000) >> 16;
            *pfb = comp;
            ++buf;
            ++pfb;
        }
    }

where it process every even lines, extract only bit 23:16 from s_dma_buf for s_fb (why?), and copy previous even line to odd line..

How could I contribute to the documentation of I2S module ?

igrr commented 7 years ago

line_width * 4 instead of line_width * 2 is because I2S peripheral captures 16 bit of data every clock cycle, even though we are only using 8 bits.

(buf_line_width - 2) * 2 should probably be (buf_line_width - margin) * bytes_per_pixel.margin is meant to skip a few pixels, so that if the first pixel that has been captured wasn't the first pixel of the scanline, we will eventually converge to the first pixel for one of the following scanlines. The bytes_per_pixel in this case is 2.

The format of data received from the camera in YUV mode looks like this (up to a permutation of components), Y is luminance component and U, V are chrominance. Chroma components are subsampled twice.

U0 Y0 V0 Y1 U2 Y2 V2 Y3 ...

Because I2S stores 16 bit of data at a time, the data s_dma_buf looks like this: U0 0 Y0 0 V0 0 Y1 0 U2 0 Y2 0 V0 0 Y3 0...

Therefore we extract 1 byte of each word to get luminance component, obtaining a grayscale image.

As for why we skip every second line... I think initially i was running this at 80 MHz so I had to skip every second line to keep up with the scan line rate. At 240 MHz this is not necessary.

krzychb commented 7 years ago

As for OV7725, basing on specification (ver. 1.3.1, August 7, 2007), I am lead to believe that YUV is one byte per pixel:

I will verify this with a scope.

igrr commented 7 years ago

422 is one of YUV subsampling modes. The numbers are not bit sizes but subsampling factors. https://en.m.wikipedia.org/wiki/Chroma_subsampling

krzychb commented 7 years ago

Traps everywhere :smile:

liubenyuan commented 7 years ago

@igrr Thanks ! I get my mt9v034 working now (at least the pattern printed using print_fb is almost right).

However in your equation (buf_line_width - margin) * bytes_per_pixel

if I change bytes_per_pixel from 2 to 1 for mt9v034 (monochrome mode, 10 bits grayscale per pixel, 8 bits are used), there is no difference regarding the console output (using print_fb)
However, if margin is 2, then line 1 will always missing (i.e., in line_filter_task)
```
if (prev_buf != -1 && prev_buf == buf_idx) {
ets_printf("! %d\n", line_count);
}
```
it will print out
```
! 1
```
every frame. But the pattern is right ! (I direct my camera toward one LED light bulb.)

margin=0 is fine, but the lines are not aligned with each other (using the camera shoot one LED light bulb, there are multiple lines unaligned with each other).

For mt9v034, if I config s_fb_w=160, then there will be exactly 160 rising edges of pclk during href=1. Is this the reason that margin should be set to 0 for this chip ?

krzychb commented 7 years ago

@liubenyuan,

While waiting for reply from @igrr, instead of checking the live image that is not stable and may be distorted of various reasons, I would configure sensor to generate a test pattern.

There is such option for OV7725 and I believe there should be one for MT9V034 - http://www.staroceans.org/myprojects/aptina-driver/MT9V034/Angstrom/mt9v034.c, search for MT9V034_TEST_PATTERN

liubenyuan commented 7 years ago

@krzychb yes, there is a test pattern register, I will try tcp/ip transfer first (save images remotely), and checkout the test pattern.

liubenyuan commented 7 years ago

@krzychb mt9v034 is targeted at application for low light, and its interface is easy to use. I plan to integrated mt9v034 with esp32, for educational purpose 😁😁

you can get one from taobao, here is a link for a newer version of the breakout board ,

http://m.intl.taobao.com/detail/detail.html?spm=a1z5f.7632060.0.0&id=19722144419

igrr / esp32-cam-demo

Some issues about DMA and I2S #7