Troubling rendering times

[lmurdock12]

So I am trying to use this library alongside the ESP32-HUB75-MatrixPanel-I2S-DMA to display scrolling images with an LED matrix. I started off by adapting the example that uses your animated GIF library to render GIF's to the matrix panel https://github.com/mrfaptastic/ESP32-HUB75-MatrixPanel-I2S-DMA/blob/master/examples/AnimatedGIFPanel/AnimatedGIFPanel.ino

I have uploaded several PNG's to flash, and using SPIFFS to open and display them. I am getting really weird rendering times for some images and I am unable to track down why. In short an 83 byte PNG is taking 57ms to render to the screen, while even an 8KB image is only taking 13ms to render. I am running the same exact code so I am completely confused. I have been using gimp to export my images with the correct dimensions with nothing bigger than a 32 pixel height.

Here is how I am testing:

    DRAW_ARGS args;
    args.xoff = 0;
    args.yoff = 0;

    long start_tick = millis();
    int res = png.open("/pngs/uparrow.png",PNGOpenFile, PNGCloseFile, PNGReadFile, PNGSeekFile, PNGRender);
    png.decode((void*)&args,PNG_FAST_PALETTE);
    int width = png.getWidth();
    png.close();

    Serial.printf("Render time up arrow: %d\n",millis() - start_tick);

    start_tick = millis();
    res = png.open("/pngs/uparrow2.png",PNGOpenFile, PNGCloseFile, PNGReadFile, PNGSeekFile, PNGRender);
    png.decode((void*)&args,PNG_FAST_PALETTE);
    width = png.getWidth();
    png.close();

    Serial.printf("Render time uparrow2: %d\n",millis() - start_tick);

    start_tick = millis();
    res = png.open("/pngs/uparrow3.png",PNGOpenFile, PNGCloseFile, PNGReadFile, PNGSeekFile, PNGRender);
    png.decode((void*)&args,PNG_FAST_PALETTE);
    width = png.getWidth();
    png.close();
    //test.renderImage();
    Serial.printf("Render time uparrow3: %d\n",millis() - start_tick);

    start_tick = millis();
    res = png.open("/pngs/blue.png",PNGOpenFile, PNGCloseFile, PNGReadFile, PNGSeekFile, PNGRender);
    png.decode((void*)&args,PNG_FAST_PALETTE);
    width = png.getWidth();
    png.close();
    //test.renderImage();
    Serial.printf("Render time blue: %d\n",millis() - start_tick);

Here are my callbacks:

typedef struct custom_draw_args
{
  int xoff, yoff; // corner offset

} DRAW_ARGS;

// Draw a line of image directly on the LED Matrix
void PNGRender(PNGDRAW *pDraw)
{

  DRAW_ARGS* drawArgs = (DRAW_ARGS*)pDraw->pUser;

  uint16_t *s;
  uint16_t usPixels[320];
  int x, y, iWidth;

  iWidth = pDraw->iWidth;
  if (iWidth > MATRIX_WIDTH)
      iWidth = MATRIX_WIDTH;

  y = pDraw->y; // current line

  // Translate the 8-bit pixels through the RGB565 palette (already byte reversed)
  //drawArgs->currPNG->getLineAsRGB565(pDraw, usPixels, PNG_RGB565_LITTLE_ENDIAN, 0xffffffff);
  png.getLineAsRGB565(pDraw, usPixels, PNG_RGB565_LITTLE_ENDIAN, 0xffffffff);
  s = usPixels;

  for (x=0; x<pDraw->iWidth; x++)
  {

    dma_display->drawPixel(drawArgs->xoff + x, drawArgs->yoff + y, *s++); // color 565
  }

}

void * PNGOpenFile(const char *fname, int32_t *pSize)
{

  f = FILESYSTEM.open(fname);

  if (f)
  {
    *pSize = f.size();
    return (void *)&f;
  }
  return NULL;
}

void PNGCloseFile(void *pHandle)
{
  File *f = static_cast<File *>(pHandle);
  if (f != NULL)
     f->close();
} 

int32_t PNGReadFile(PNGFILE *pFile, uint8_t *pBuf, int32_t iLen)
{
    int32_t iBytesRead;
    iBytesRead = iLen;
    File *f = static_cast<File *>(pFile->fHandle);
    // Note: If you read a file all the way to the last byte, seek() stops working
    if ((pFile->iSize - pFile->iPos) < iLen)
       iBytesRead = pFile->iSize - pFile->iPos - 1; // <-- ugly work-around
    if (iBytesRead <= 0)
       return 0;
    iBytesRead = (int32_t)f->read(pBuf, iBytesRead);
    pFile->iPos = f->position();
    return iBytesRead;
} 

int32_t PNGSeekFile(PNGFILE *pFile, int32_t iPosition)
{ 
  int i = micros();
  File *f = static_cast<File *>(pFile->fHandle);
  f->seek(iPosition);
  pFile->iPos = (int32_t)f->position();
  i = micros() - i;
  return pFile->iPos;
} 

Here are the properties of the 4 images I am testing:

1. uparrow.png -- 721 bytes, 21x20. This is a green arrow
2. uparrow2.png -- 8kb, 21x20. This is the same png but without compressing size down
3. uparrow3.png -- 151 bytes, 21x20. Same arrow but in a blue color
4. blue.png -- 83 bytes, 15x9. This is as simple as it gets: a blue rectangle.

Here are the runtimes I am currently getting:

Render time uparrow: 6 Render time uparrow2: 13 Render time uparrow3: 3 Render time blue: 57

Why is the smallest image taking the longest time to render? Is there some sort of specific settings I am missing when exporting my PNG from gimp? 57ms is drastically longer than everything else.

Your decoding test is extremely quick so something weird is going on.

If there is a more optimal way I should be using this library please let me know!

Here is a link to my zipped platformIO project that includes the actual images: https://drive.google.com/file/d/11PT4aziMz-JiPMY_gxZDcFyuHViaZgTn/view?usp=sharing

[bitbank2]

Calling drawpixel() on each pixel in the PNGDRAW callback means that you're not really measuring the time to decode the PNG. Try removing that call and see how much time it takes.

[lmurdock12]

I reduced my render call back function to the following and I am still getting the same results:

void PNGRender(PNGDRAW *pDraw)
{
  return; 
}

################# STARTING DISPLAY ################# Render time up arrow: 6 Render time uparrow2: 13 Render time uparrow3: 3 Render time blue: 56

Something really weird is going on here...

[lmurdock12]

Added more time debug statements and the Open callback is the issue

################# STARTING DISPLAY ################# ----> Open: 3 ----> Read: 1 ----> Seek: 0 ----> Read: 0 ----> Seek: 0 ----> Close: 0 Render time up arrow: 7 ----> Open: 1 ----> Read: 3 ----> Seek: 0 ----> Read: 3 ----> Seek: 1 ----> Read: 2 ----> Seek: 0 ----> Read: 0 ----> Seek: 2 ----> Close: 0 Render time uparrow2: 14 ----> Open: 1 ----> Read: 1 ----> Seek: 0 ----> Read: 0 ----> Seek: 0 ----> Close: 0 Render time uparrow3: 14 ----> Open: 55 ----> Read: 1 ----> Seek: 0 ----> Read: 0 ----> Seek: 0 ----> Close: 0 Render time blue: 57

void * PNGOpenFile(const char *fname, int32_t *pSize)
{

    long start = millis();
    f = FILESYSTEM.open(fname);

    if (f)
    {
    *pSize = f.size();
    Serial.printf("----> Open: %d\n",millis() - start);
    return (void *)&f;
    }
    return NULL;
}

So this is a SPIFFS issue it seems...

If this delay is basically unavoidable with Spiffs I am going to have to look for some sort of alternative.

I need to render 4-5 images at a time preferably under 40ms. The best solution for me would be to handle the overhead of opening the files I need earlier before I get to the main rendering loops. Is there any way to do this with the library outside of modifying the open call back arguments to allow me to pass in and return an already opened file?

Storing the bytes in a .h file and using openFLASH() or openRAM() will not work for me here as I will run into limitations with the amount of memory I have to store images.

[bitbank2]

That's good detective work. I've used SPIFFS before and saw some perf hiccups, but didn't investigate it further. I'm not sure why you're going to run into issues with storage limitations because SPIFFS = FLASH. If you move the images into .H files, you're using the same physical memory to store them. You can just change the FLASH partitioning to make your app area larger and the SPIFFS area smaller.

[lmurdock12]

That's good detective work. I've used SPIFFS before and saw some perf hiccups, but didn't investigate it further. I'm not sure why you're going to run into issues with storage limitations because SPIFFS = FLASH. If you move the images into .H files, you're using the same physical memory to store them. You can just change the FLASH partitioning to make your app area larger and the SPIFFS area smaller.

I need to dynamically be able to store images and then read and display them quickly. I don't think the .h way would work in that instance?

[bitbank2]

yes, dynamic changing of FLASH should probably be done through the SPIFFS API. I guess your other option is to use the second core of the ESP32 to cache them in RAM right before you use them so that a SPIFFS hiccup won't delay your main display thread.