platformio / platform-espressif32

Espressif 32: development platform for PlatformIO
https://registry.platformio.org/platforms/platformio/espressif32
Apache License 2.0
928 stars 629 forks source link

LittleFS fails on LOLIN_S3 board if you set board = lolin_s3 #1226

Open mars000 opened 1 year ago

mars000 commented 1 year ago

What kind of issue is this?

I have a LOLIN S3 board. In platformio.ini file, when I compile below code using board = lolin_s3and it fails to run/mount LittleFS. If I simply change board = lolin_s3_mini all works perferctly.


Configuration

Windows 11

PlatformIO Version (platformio --version):

Description of problem

Fails when board = lolin_s3 but works perfectly when board = lolin_s3_mini in platformio.inifile My actual board is LOLIN_S3 for all tests.

platformio.ini file:

[env:lolin_s3]
platform = espressif32@6.4.0
board = lolin_s3
framework = arduino
board_build.filesystem = littlefs
upload_port = COM8
monitor_speed = 115200

Serial Output when board = lolin_s3 in platformio.ini file

---- Opened the serial port COM8 ----
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4
[   138][E][vfs_api.cpp:332] VFSFileImpl(): fopen(/littlefs/mydir/newdir2/newdir3/hello3.txt) failed
[   140][E][vfs_api.cpp:105] open(): /littlefs/mydir/newdir2/newdir3/hello3.txt does not exist, no permits for creation
[   227][E][vfs_api.cpp:29] open(): mydir does not start with /
[   254][E][vfs_api.cpp:182] remove(): /mydir/newdir2/newdir3/hello3.txt does not exists or is directory
[   283][E][vfs_api.cpp:250] rmdir(): /mydir does not exists or is a file

Serial Output when board = lolin_s3_mini in platformio.ini file

---- Opened the serial port COM8 ----
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4
E (93) psram: PSRAM ID read error: 0x00ffffff, PSRAM chip not found or not���LittleFS Mount OK!
Listing directory: /
Creating Dir: /mydir
Dir created
Writing file: /mydir/hello2.txt
- file written
Writing file: /mydir/newdir2/newdir3/hello3.txt
[   158][E][vfs_api.cpp:332] VFSFileImpl(): fopen(/littlefs/mydir/newdir2/newdir3/hello3.txt) failed
- failed to open file for writing
[   172][E][vfs_api.cpp:105] open(): /littlefs/mydir/newdir2/newdir3/hello3.txt does not exist, no permits for creation
Create missing folders of: /mydir/newdir2/newdir3/hello3.txt
Writing file to: /mydir/newdir2/newdir3/hello3.txt
- file written
Listing directory: /
  DIR : mydir  LAST WRITE: 1969-12-31 23:59:59
Listing directory: mydir
[   286][E][vfs_api.cpp:29] open(): mydir does not start with /
- failed to open directory
Deleting file: /mydir/hello2.txt
- file deleted
Deleting file: /mydir/newdir2/newdir3/hello3.txt
- file deleted
Deleting file and empty folders on path: /mydir/newdir2/newdir3/hello3.txt
[   328][E][vfs_api.cpp:182] remove(): /mydir/newdir2/newdir3/hello3.txt does not exists or is directory
- delete failed
Removing all empty folders on path: /mydir/newdir2/newdir3/hello3.txt
Removing Dir: /mydir
[   380][E][vfs_api.cpp:250] rmdir(): /mydir does not exists or is a file
rmdir failed
Listing directory: /
Writing file: /hello.txt
- file written
Appending to file: /hello.txt
- message appended
Reading file: /hello.txt
- read from file:
Hello World!
Renaming file /hello.txt to /foo.txt
- file renamed
Reading file: /foo.txt
- read from file:
Hello World!
Deleting file: /foo.txt
- file deleted
Testing file I/O with /test.txt
- writing................................................................
 - 1048576 bytes written in 11928 ms
- reading................................................................
- 1048576 bytes read in 349 ms
Deleting file: /test.txt
- file deleted
Test complete

Source code: main.cpp

#include <Arduino.h>

#include "FS.h"
#include "LittleFS.h"

#include <time.h>

/* You only need to format LittleFS the first time you run a
   test or else use the LITTLEFS plugin to create a partition
   https://github.com/lorol/arduino-esp32littlefs-plugin */

#define FORMAT_LITTLEFS_IF_FAILED true

void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\r\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("- failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println(" - not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");

            Serial.print(file.name());
            time_t t= file.getLastWrite();
            struct tm * tmstruct = localtime(&t);
            Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n",(tmstruct->tm_year)+1900,( tmstruct->tm_mon)+1, tmstruct->tm_mday,tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

            if(levels){
                listDir(fs, file.name(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");

            Serial.print(file.size());
            time_t t= file.getLastWrite();
            struct tm * tmstruct = localtime(&t);
            Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n",(tmstruct->tm_year)+1900,( tmstruct->tm_mon)+1, tmstruct->tm_mday,tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);
        }
        file = root.openNextFile();
    }
}

void createDir(fs::FS &fs, const char * path){
    Serial.printf("Creating Dir: %s\n", path);
    if(fs.mkdir(path)){
        Serial.println("Dir created");
    } else {
        Serial.println("mkdir failed");
    }
}

void removeDir(fs::FS &fs, const char * path){
    Serial.printf("Removing Dir: %s\n", path);
    if(fs.rmdir(path)){
        Serial.println("Dir removed");
    } else {
        Serial.println("rmdir failed");
    }
}

void readFile(fs::FS &fs, const char * path){
    Serial.printf("Reading file: %s\r\n", path);

    File file = fs.open(path);
    if(!file || file.isDirectory()){
        Serial.println("- failed to open file for reading");
        return;
    }

    Serial.println("- read from file:");
    while(file.available()){
        Serial.write(file.read());
    }
    file.close();
}

void writeFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Writing file: %s\r\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("- failed to open file for writing");
        return;
    }
    if(file.print(message)){
        Serial.println("- file written");
    } else {
        Serial.println("- write failed");
    }
    file.close();
}

void appendFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Appending to file: %s\r\n", path);

    File file = fs.open(path, FILE_APPEND);
    if(!file){
        Serial.println("- failed to open file for appending");
        return;
    }
    if(file.print(message)){
        Serial.println("- message appended");
    } else {
        Serial.println("- append failed");
    }
    file.close();
}

void renameFile(fs::FS &fs, const char * path1, const char * path2){
    Serial.printf("Renaming file %s to %s\r\n", path1, path2);
    if (fs.rename(path1, path2)) {
        Serial.println("- file renamed");
    } else {
        Serial.println("- rename failed");
    }
}

void deleteFile(fs::FS &fs, const char * path){
    Serial.printf("Deleting file: %s\r\n", path);
    if(fs.remove(path)){
        Serial.println("- file deleted");
    } else {
        Serial.println("- delete failed");
    }
}

// LittleFS-like write and delete file

// See: https://github.com/esp8266/Arduino/blob/master/libraries/LittleFS/src/LittleFS.cpp#L60
void writeFile2(fs::FS &fs, const char * path, const char * message){
    if(!fs.exists(path)){
        if (strchr(path, '/')) {
            Serial.printf("Create missing folders of: %s\r\n", path);
            char *pathStr = strdup(path);
            if (pathStr) {
                char *ptr = strchr(pathStr, '/');
                while (ptr) {
                    *ptr = 0;
                    fs.mkdir(pathStr);
                    *ptr = '/';
                    ptr = strchr(ptr+1, '/');
                }
            }
            free(pathStr);
        }
    }

    Serial.printf("Writing file to: %s\r\n", path);
    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("- failed to open file for writing");
        return;
    }
    if(file.print(message)){
        Serial.println("- file written");
    } else {
        Serial.println("- write failed");
    }
    file.close();
}

// See:  https://github.com/esp8266/Arduino/blob/master/libraries/LittleFS/src/LittleFS.h#L149
void deleteFile2(fs::FS &fs, const char * path){
    Serial.printf("Deleting file and empty folders on path: %s\r\n", path);

    if(fs.remove(path)){
        Serial.println("- file deleted");
    } else {
        Serial.println("- delete failed");
    }

    char *pathStr = strdup(path);
    if (pathStr) {
        char *ptr = strrchr(pathStr, '/');
        if (ptr) {
            Serial.printf("Removing all empty folders on path: %s\r\n", path);
        }
        while (ptr) {
            *ptr = 0;
            fs.rmdir(pathStr);
            ptr = strrchr(pathStr, '/');
        }
        free(pathStr);
    }
}

void testFileIO(fs::FS &fs, const char * path){
    Serial.printf("Testing file I/O with %s\r\n", path);

    static uint8_t buf[512];
    size_t len = 0;
    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("- failed to open file for writing");
        return;
    }

    size_t i;
    Serial.print("- writing" );
    uint32_t start = millis();
    for(i=0; i<2048; i++){
        if ((i & 0x001F) == 0x001F){
          Serial.print(".");
        }
        file.write(buf, 512);
    }
    Serial.println("");
    uint32_t end = millis() - start;
    Serial.printf(" - %u bytes written in %u ms\r\n", 2048 * 512, end);
    file.close();

    file = fs.open(path);
    start = millis();
    end = start;
    i = 0;
    if(file && !file.isDirectory()){
        len = file.size();
        size_t flen = len;
        start = millis();
        Serial.print("- reading" );
        while(len){
            size_t toRead = len;
            if(toRead > 512){
                toRead = 512;
            }
            file.read(buf, toRead);
            if ((i++ & 0x001F) == 0x001F){
              Serial.print(".");
            }
            len -= toRead;
        }
        Serial.println("");
        end = millis() - start;
        Serial.printf("- %u bytes read in %u ms\r\n", flen, end);
        file.close();
    } else {
        Serial.println("- failed to open file for reading");
    }
}

void setup(){
    Serial.begin(115200);
    if (!LittleFS.begin(true)){
        Serial.println("LittleFS Mount Failed");
        return;
    } else {
        Serial.println("LittleFS Mount OK!");
    }

    listDir(LittleFS, "/", 0);
    createDir(LittleFS, "/mydir");
    writeFile(LittleFS, "/mydir/hello2.txt", "Hello2");
    writeFile(LittleFS, "/mydir/newdir2/newdir3/hello3.txt", "Hello3");
    writeFile2(LittleFS, "/mydir/newdir2/newdir3/hello3.txt", "Hello3");
    listDir(LittleFS, "/", 3);
    deleteFile(LittleFS, "/mydir/hello2.txt");
    deleteFile(LittleFS, "/mydir/newdir2/newdir3/hello3.txt");
    deleteFile2(LittleFS, "/mydir/newdir2/newdir3/hello3.txt");
    removeDir(LittleFS, "/mydir");
    listDir(LittleFS, "/", 3);
    writeFile(LittleFS, "/hello.txt", "Hello ");
    appendFile(LittleFS, "/hello.txt", "World!\r\n");
    readFile(LittleFS, "/hello.txt");
    renameFile(LittleFS, "/hello.txt", "/foo.txt");
    readFile(LittleFS, "/foo.txt");
    deleteFile(LittleFS, "/foo.txt");
    testFileIO(LittleFS, "/test.txt");
    deleteFile(LittleFS, "/test.txt");

    Serial.println( "Test complete" );
}

void loop(){

}
valeros commented 1 year ago

Hi @mars000, the only obvious difference is the partition table, lolin_s3_mini uses the default partition while lolin_s3 uses a special default_16MB. What happens if you manually set your partition to the default one directly in the platformio.ini file, for example:

[env:lolin_s3]
platform = espressif32@6.4.0
board = lolin_s3
framework = arduino
board_build.filesystem = littlefs
upload_port = COM8
monitor_speed = 115200

board_build.partitions = default.csv
mars000 commented 1 year ago
board_build.partitions = default.csv

hi @valeros I updated and added above suggestion i.e. board_build.partitions = default.csv but no change i.e. same serial output as I placed in original post above is displayed.

valeros commented 1 year ago

Have you tried erasing the entire flash memory? Just run pio run -t erase directly in the IDE terminal.

mars000 commented 11 months ago

thank @valeros just tried that and it did't change the output.

robertlipe commented 11 months ago

(flyby - I don't have the hardware involved - I'm just reading code...)

There's another difference above. One has working PSRAM and one doesn't. That MIGHT imply a misconfigured SPI clock as the SPI is used for both FLASH and RAM.

https://www.wemos.cc/en/latest/s3/s3.html Shows QSPI Flash and OSPI RAM https://www.wemos.cc/en/latest/s3/s3_mini.html is rather less detailed, but looks like a "value" board, so it may be all QSPI or even DSPI.

It's possible that setting it to the Mini uses a more conservative SPI configuration that happens to word. Maybe you have something else using a GPIO pin that's used in Octal mode, for example....

I'm talking out of my hat a little bit, having never seen these boards, but I've debugged a lot of slightly wrong ESP PSRAM configurations...

I suspect that LittleFS vs. SPIFFS is a victim and not a culprit here.

Good luck!

mars000 commented 11 months ago

thanks @robertlipe for insights. Would that imply a configurstion issue in platformio profiles ? I'm just using standard config as per docs.

robertlipe commented 11 months ago

Could be Platformio or even esp-idf. Could be the board maker quietly changed the specs and a config that used to work doesn't.

I'm just suggesting that you widen the scope of your hunt beyond littlefs (spiffs probably also misbehaves) into the SPI configuration. I'm not at all sure they're interchangeable, but I am sure that a correctly configured board with psram won't print that message you show about psraminit failing and that may be within blast radius of why the filesystem (likely actually the underlying SPI accesses) are failing.

I could be wrong.

mars000 commented 11 months ago

thank you @robertlipe @ivankravets your thoughts on this would be greatly appreciated

Jason2866 commented 11 months ago

The main difference of the two boards json are the type of the PSRAM flash. S3 mini has QSPI and the Lolin S3 has OPI. This looks correct according to the description from the Lolin we page. Maybe in real the Lolin S3 has no OPI PSRAM? Can be verified by the mounted module on the board. Upload an highres picture and we can check.

Wrong PSRAM selection does generate Issues with Filesystem. The s3mini boards json does work since no valid PSRAM is found and it works as S3 without any PSRAM

mars000 commented 11 months ago

@Jason2866 here is hi-res picture, hope it helps answer question IMG_0206

Jason2866 commented 11 months ago

Mhh, the last line looks like "N16R8" regarding to the datasheet it is 16MB QSPI Flash and 8MB OPI PSRAM PR #1252 removes wrong entrys.

mars000 commented 11 months ago

hello @valeros and @Jason2866 I copied your lolin_s3.json file manually (not sure how else to best updates platfromio in vsc with your changes) with your updates as noted in https://github.com/platformio/platform-espressif32/issues/1226 but still get same errors :-( Something i'm missing...this is the lolin_s3.json file

`{ "build": { "arduino": { "ldscript": "esp32s3_out.ld", "partitions": "default_16MB.csv", "memory_type": "qio_opi" }, "core": "esp32", "extra_flags": [ "-DARDUINO_LOLIN_S3", "-DBOARD_HAS_PSRAM", "-DARDUINO_USB_MODE=1", "-DARDUINO_USB_CDC_ON_BOOT=1", "-DARDUINO_RUNNING_CORE=1", "-DARDUINO_EVENT_RUNNING_CORE=1" ], "f_cpu": "240000000L", "f_flash": "80000000L", "flash_mode": "qio", "hwids": [ [ "0x303A", "0x1001" ] ], "mcu": "esp32s3", "variant": "lolin_s3" }, "connectivity": [ "wifi", "bluetooth" ], "debug": { "openocd_target": "esp32s3.cfg" }, "frameworks": [ "arduino", "espidf" ], "name": "WEMOS LOLIN S3", "upload": { "flash_size": "16MB", "maximum_ram_size": 327680, "maximum_size": 16777216, "require_upload_port": true, "speed": 460800 }, "url": "https://www.wemos.cc/en/latest/s3/index.html", "vendor": "WEMOS" }

robertlipe commented 11 months ago

Json's theory was: "Maybe in real the Lolin S3 has no OPI PSRAM?"

Board vendors change the actual parts splashed down on parts All The Time. It's terrible, but it happens. To test that theory ...

"memorytype": "qioopi"

I'm honestly not certain which side trumps the other here, but hold that thought...

"flash_mode": "qio",

This sets the flash mode to quad. Flash on ESP32 is never > Q. I don't see a line following this like "psram_type": "opi", In theory, this would set it to octal. Experiment with adding that and/or changing it to "qio" (maybe qpi - it's hard to find accurate doc here because we're deep into the intersection of the bootloader, esptool., the boot on the chip, the signal configuration, what' actually on the board, how the various fuses are blown, the possibility of slightly broken boards, etc.

I don't have answers. I'm just telling you what you'd see if you were looking over my shoulder while I were trying to hit and miss this board - like I've had to do with so many other ESP32-S3's. It's a mess.

Citation: https://community.platformio.org/t/esp32-s3-devkitc-1-n32r8v-configuration-assistance/31967/3

Be prepared to dumpster-dive around in files inside https://github.com/platformio/platform-espressif32/tree/develop/boards ...particulularly those with "S3" in their names or descriptions.

Good luck!

RJL

Jason2866 commented 11 months ago

@robertlipe There is some code to take care that correct and needed flash modes are set in Platformio here https://github.com/platformio/platform-espressif32/blob/develop/builder/main.py#L45-L86 My guess, the board PSRAM is defect.

I have all S3 variants of flash / PSRAM boards. The all work well. BUT it is mandatory that every detail is correct set!

mars000 commented 11 months ago

Hi @valeros @Jason2866 @robertlipe

don't think its a hardware fault on my lolin s3 board, as the exact same example code works perfectly when using Arduino IDE. Something going on in platformio config ?? Anyway, I managed to get it going by editing/hacking thelolin_s3.json file. It seems it work with the lolin_s3.json file pasted below. any insights would be much appreciated as i know these lolin s3 cards are very popular so i'm sure others will fall into these issues as some point.

you will notice all i did is remove the following lines in "extra_flags"

 "-DARDUINO_USB_CDC_ON_BOOT=1",
 "-DARDUINO_RUNNING_CORE=1",
 "-DARDUINO_EVENT_RUNNING_CORE=1"

Working lolin_s3.json file:

{
  "build": {
    "arduino": {
      "ldscript": "esp32s3_out.ld",
      "partitions": "default_16MB.csv",
      "memory_type": "qio_opi"
    },
    "core": "esp32",
    "extra_flags": [
      "-DARDUINO_LOLIN_S3",
      "-DBOARD_HAS_PSRAM",
      "-DARDUINO_USB_MODE=1"
    ],
    "f_cpu": "240000000L",
    "f_flash": "80000000L",
    "flash_mode": "qio",
    "hwids": [
      [
        "0x303A",
        "0x1001"
      ]
    ],
    "mcu": "esp32s3",
    "variant": "lolin_s3"
  },
  "connectivity": [
    "wifi",
    "bluetooth"
  ],
  "debug": {
    "openocd_target": "esp32s3.cfg"
  },
  "frameworks": [
    "arduino",
    "espidf"
  ],
  "name": "WEMOS LOLIN S3",
  "upload": {
    "flash_size": "16MB",
    "maximum_ram_size": 327680,
    "maximum_size": 16777216,
    "require_upload_port": true,
    "speed": 460800
  },
  "url": "https://www.wemos.cc/en/latest/s3/index.html",
  "vendor": "WEMOS"
}
stale[bot] commented 8 months ago

This issue has been automatically marked as stale because it has not had recent activity. Please provide more details or it will be closed if no further activity occurs. Thank you for your contributions.