Unable to create a LittleFS partition with default or custom ESP32 partition schemes when 32MB flash installed

[pauleilio]

Board

ESP32-S3R8 SoC custom board

Device Description

Own hardware based on the ESP32-S3R8 SoC with 32MB Flash and 8MB OCTAL Psram

Hardware Configuration

RTC Clock IO Expander RGB Display Touch controller

Version

v2.0.9

IDE Name

Arduino IDE

Operating System

Windows 11

Flash frequency

80Mhz

PSRAM enabled

yes

Upload speed

921600

Description

Unable to mount LittleFS Filesystem using default ESP32-S3 Dev Board partition schemes when 32MB Flash is installed. I have tried all partition schemes and Ffat partition schemes and tested with Ffat library test example and LittleFS library test example. Both report Ffat / LittleFS mount failed. I also Tried a custom partition csv which worked fine with everything else apart from Ffat and LittleFS.

Board works perfectly with Display, IOExpander, Touch, RTC and psram works without a problem as a framebuffer with the display.

Issue is still present with Psram disabled.

I use MX25L25645GM2I-08G and our Espressif contact has said MX25L25645GM2I-08G was tested succesfully at 80Mhz but not 120Mhz, and have been using 80Mhz successfully for all other functions, its just this LittleFS etc which does not work.

I followed this tutorial but there is no 32MB example

https://github.com/espressif/arduino-esp32/blob/master/docs/source/tutorials/partition_table.rst

I also tried the partition generator from here with the same issue

https://github.com/espressif/arduino-esp32/issues/6495#issuecomment-1111177668

I have looked for solutions online but there is very little information available for a 32MB flash setup

Sketch

#include "FS.h"
#include "FFat.h"

// This file should be compiled with 'Partition Scheme' (in Tools menu)
// set to 'Default with ffat' if you have a 4MB ESP32 dev module or
// set to '16M Fat' if you have a 16MB ESP32 dev module.

// You only need to format FFat the first time you run a test
#define FORMAT_FFAT true

void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\r\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("- failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println(" - not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                listDir(fs, file.path(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("\tSIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void readFile(fs::FS &fs, const char * path){
    Serial.printf("Reading file: %s\r\n", path);

    File file = fs.open(path);
    if(!file || file.isDirectory()){
        Serial.println("- failed to open file for reading");
        return;
    }

    Serial.println("- read from file:");
    while(file.available()){
        Serial.write(file.read());
    }
    file.close();
}

void writeFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Writing file: %s\r\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("- failed to open file for writing");
        return;
    }
    if(file.print(message)){
        Serial.println("- file written");
    } else {
        Serial.println("- write failed");
    }
    file.close();
}

void appendFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Appending to file: %s\r\n", path);

    File file = fs.open(path, FILE_APPEND);
    if(!file){
        Serial.println("- failed to open file for appending");
        return;
    }
    if(file.print(message)){
        Serial.println("- message appended");
    } else {
        Serial.println("- append failed");
    }
    file.close();
}

void renameFile(fs::FS &fs, const char * path1, const char * path2){
    Serial.printf("Renaming file %s to %s\r\n", path1, path2);
    if (fs.rename(path1, path2)) {
        Serial.println("- file renamed");
    } else {
        Serial.println("- rename failed");
    }
}

void deleteFile(fs::FS &fs, const char * path){
    Serial.printf("Deleting file: %s\r\n", path);
    if(fs.remove(path)){
        Serial.println("- file deleted");
    } else {
        Serial.println("- delete failed");
    }
}

void testFileIO(fs::FS &fs, const char * path){
    Serial.printf("Testing file I/O with %s\r\n", path);

    static uint8_t buf[512];
    size_t len = 0;
    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("- failed to open file for writing");
        return;
    }

    size_t i;
    Serial.print("- writing" );
    uint32_t start = millis();
    for(i=0; i<2048; i++){
        if ((i & 0x001F) == 0x001F){
          Serial.print(".");
        }
        file.write(buf, 512);
    }
    Serial.println("");
    uint32_t end = millis() - start;
    Serial.printf(" - %u bytes written in %u ms\r\n", 2048 * 512, end);
    file.close();

    file = fs.open(path);
    start = millis();
    end = start;
    i = 0;
    if(file && !file.isDirectory()){
        len = file.size();
        size_t flen = len;
        start = millis();
        Serial.print("- reading" );
        while(len){
            size_t toRead = len;
            if(toRead > 512){
                toRead = 512;
            }
            file.read(buf, toRead);
            if ((i++ & 0x001F) == 0x001F){
              Serial.print(".");
            }
            len -= toRead;
        }
        Serial.println("");
        end = millis() - start;
        Serial.printf("- %u bytes read in %u ms\r\n", flen, end);
        file.close();
    } else {
        Serial.println("- failed to open file for reading");
    }
}

void setup(){
    Serial.begin(115200);
    delay(5000);
    Serial.println("testing FFAT");
    Serial.setDebugOutput(true);
    if (FORMAT_FFAT) FFat.format();
    if(!FFat.begin()){
        Serial.println("FFat Mount Failed");
        return;
    }

    Serial.printf("Total space: %10u\n", FFat.totalBytes());
    Serial.printf("Free space: %10u\n", FFat.freeBytes());
    listDir(FFat, "/", 0);
    writeFile(FFat, "/hello.txt", "Hello ");
    appendFile(FFat, "/hello.txt", "World!\r\n");
    readFile(FFat, "/hello.txt");
    renameFile(FFat, "/hello.txt", "/foo.txt");
    readFile(FFat, "/foo.txt");
    deleteFile(FFat, "/foo.txt");
    testFileIO(FFat, "/test.txt");
    Serial.printf("Free space: %10u\n", FFat.freeBytes());
    deleteFile(FFat, "/test.txt");
    Serial.println( "Test complete" );
}

void loop(){

}

Debug Message

3:21:38.867 -> testing FFAT
23:21:39.849 -> FFat Mount Failed

23:36:17.706 -> ./components/esp_littlefs/src/littlefs/lfs.c:1886:debug: Bad block at 0x0
23:36:17.706 -> ./components/esp_littlefs/src/littlefs/lfs.c:1892:warn: Superblock 0x0 has become unwritable
23:36:17.706 -> E (6844) esp_littlefs: Failed to format filesystem
23:36:17.706 -> LittleFS Mount Failed

Other Steps to Reproduce

The same setup with 4 or 16MB Flash worked as expected with both Ffat and LittleFS examples finishing the test successfully

I have checked existing issues, online documentation and the Troubleshooting Guide

• [X] I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[lbernstone]

Can you please test with an IDF example? If this is an issue with the upstream components, there is little that can be done in arduino-esp32.

[pauleilio]

This is the ouptut from ESP-IDF 4.4

I (0) cpu_start: App cpu up. I (203) cpu_start: Pro cpu start user code I (203) cpu_start: cpu freq: 160000000 I (203) cpu_start: Application information: I (206) cpu_start: Project name: wear_levelling_example I (212) cpu_start: App version: v4.4.4-dirty I (218) cpu_start: Compile time: Jun 30 2023 08:00:27 I (224) cpu_start: ELF file SHA256: 74e01b3087c8f95e... I (230) cpu_start: ESP-IDF: v4.4.4-dirty I (235) heap_init: Initializing. RAM available for dynamic allocation: I (242) heap_init: At 3FC94CB8 len 00054A58 (338 KiB): D/IRAM I (249) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM I (255) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM I (261) heap_init: At 600FE000 len 00002000 (8 KiB): RTCRAM I (268) spi_flash: detected chip: mxic W (272) spi_flash: Detected flash size > 16 MB, but access beyond 16 MB is not supported for this flash model yet. I (284) spi_flash: flash io: dio W (287) spi_flash: Detected size(32768k) larger than the size in the binary image header(4096k). Using the size in the binary image header. I (301) sleep: Configure to isolate all GPIO pins in sleep state I (307) sleep: Enable automatic switching of GPIO sleep configuration I (314) cpu_start: Starting scheduler on PRO CPU. I (0) cpu_start: Starting scheduler on APP CPU. I (335) example: Mounting FAT filesystem W (495) vfs_fat_spiflash: f_mount failed (13) I (495) vfs_fat_spiflash: Formatting FATFS partition, allocation unit size=4096 I (695) vfs_fat_spiflash: Mounting again I (695) example: Opening file E (695) example: Failed to open file for writing

[pauleilio]

This is output from ESP-IDF 5.0.1

I (0) cpu_start: App cpu up. I (191) cpu_start: Pro cpu start user code I (191) cpu_start: cpu freq: 160000000 Hz I (191) cpu_start: Application information: I (194) cpu_start: Project name: wear_levelling_example I (200) cpu_start: App version: v5.0.1 I (205) cpu_start: Compile time: Jun 30 2023 09:30:19 I (211) cpu_start: ELF file SHA256: db177b1b7614bfd3... I (217) cpu_start: ESP-IDF: v5.0.1 I (222) cpu_start: Min chip rev: v0.0 I (227) cpu_start: Max chip rev: v0.99 I (232) cpu_start: Chip rev: v0.1 I (237) heap_init: Initializing. RAM available for dynamic allocation: I (244) heap_init: At 3FC94548 len 000551C8 (340 KiB): D/IRAM I (250) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM I (257) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM I (263) heap_init: At 600FE010 len 00001FF0 (7 KiB): RTCRAM I (270) spi_flash: detected chip: mxic W (274) spi_flash: Detected flash size > 16 MB, but access beyond 16 MB is not supported for this flash model yet. I (285) spi_flash: flash io: dio W (289) spi_flash: Detected size(32768k) larger than the size in the binary image header(4096k). Using the size in the binary image header. I (303) cpu_start: Starting scheduler on PRO CPU. I (0) cpu_start: Starting scheduler on APP CPU. I (323) example: Mounting FAT filesystem I (483) example: Opening file I (663) example: File written I (663) example: Reading file I (663) example: Read from file: 'written using ESP-IDF v5.0.1' I (663) example: Unmounting FAT filesystem I (763) example: Done

[lbernstone]

That seems pretty clear. access beyond 16MB is not supported for this flash model yet. Support for the flash was added in IDF 5 (which will be used in arduino-esp32 3.x)

[James-4DSystems]

Has there been any progress on this issue, even using the 3.0.0 Alpha 3 release?

[lbernstone]

Works fine on 3.0. Make sure to choose the right size and speed flash. I made a menu entry for esp32-s3-wroom-2 (octal)

16:09:41.999 -> ------------------------------------------
16:09:41.999 -> Software Info:
16:09:41.999 -> ------------------------------------------
16:09:41.999 ->   Compile Date/Time : Dec 13 2023 16:09:27
16:09:42.032 ->   Compile Host OS   : linux
16:09:42.032 ->   ESP-IDF Version   : v5.1.1-686-gb6a66b7d8c-dirty
16:09:42.032 ->   Arduino Version   : 3.0.0
16:09:42.032 -> ------------------------------------------
16:09:42.032 -> Board Info:
16:09:42.032 -> ------------------------------------------
16:09:42.032 ->   Arduino Board     : ESP32S3_DEV
16:09:42.032 ->   Arduino Variant   : esp32s3
16:09:42.032 ->   Arduino FQBN      : espressif:esp32:esp32s3-octal:JTAGAdapter=default,PSRAM=opi,FlashMode=opi,FlashSize=32M,LoopCore=1,EventsCore=1,USBMode=hwcdc,CDCOnBoot=default,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,PartitionScheme=app5M_fat24M_32MB,CPUFreq=240,UploadSpeed=921600,DebugLevel=verbose,EraseFlash=none
16:09:42.065 -> ============ Before Setup End ============
16:09:42.164 -> [   666][V][esp32-hal-uart.c:409] uartBegin(): UART0 baud(115200) Mode(800001c) rxPin(44) txPin(43)
16:09:42.796 -> Total space:   23756800
16:09:42.796 -> Free space:   23756800

[VojtechBartoska]

Hello, closing this as expired due to no answer from author. You can reopen if needed. Thanks

[pauleilio]

Confirmed that it works fine in 3.0.0 with 32MB flash