lorol commented 4 years ago

Updates:

Combined plugin for SPIFFS, LittleFS and FatFS: https://github.com/me-no-dev/ESPAsyncWebServer/issues/792#issuecomment-658421741
For FAT partition, see https://github.com/me-no-dev/ESPAsyncWebServer/issues/792#issuecomment-656637372

Hi I made a LittleFS for ESP32, too. see the example my library and tool plus the core conversation. Please test and comment. (performance, speed, stability, code size, slashes, directories and root / as well as other compatibility concerns)

Example (modified from the master ESP_AsyncFSBrowser) Note I did tweaks of Editor as well to handle folders and slashes https://github.com/lorol/ESPAsyncWebServer/tree/master/examples/ESP_AsyncFSBrowser

Library: https://github.com/lorol/LITTLEFS

Tool: https://github.com/lorol/arduino-esp32littlefs-plugin

Core discussion: https://github.com/espressif/arduino-esp32/issues/3765

Simple usage:

include

define SPIFFS LITTLEFS

include

BlueAndi commented 4 years ago

Could you add a library.json too for platformio?

lorol commented 4 years ago

@BlueAndi - Done, just added an improvised one. I have no idea if it makes sense. Please check and let me know.

BlueAndi commented 4 years ago

A first try showed:

The "esp_littlefs.h" includes the "sys/termios.h" which contains baudrate preprocessor defines that clashes with the binary number preprocessor defines of "binary.h" (included in Arduino.h).

I see you need at least IDF v3.3 based stuff, because e.g. the esp_vfs_t doesn't match at my side currently. It looks like I still get IDF v3.2 via platformio. :-(

Edit: Yes, v3.2 is used by v1.0.4 (https://github.com/espressif/arduino-esp32/releases)

lorol commented 4 years ago

The IDF Arduino is compiled on is this v.3.3: git checkout 46b12a560a29fa6ade07800a4abe12a026183988 The original wrapper https://github.com/joltwallet/esp_littlefs was modified not long ago so it likely is tested on even newer IDF. If PIO uses older IDF, you need to resolve it yourself. My target was to work on Arduino IDE. For me, PIO looks like a F1 car we try to drive with your home car license :)

On Wed, Jun 17, 2020 at 3:02 PM Andreas Merkle notifications@github.com wrote:

A first try showed:

The "esp_littlefs.h" includes the "sys/termios.h" which contains baudrate preprocessor defines that clashes with the binary number preprocessor defines of "binary.h" (included in Arduino.h).

I see you need at least IDF v3.3 based stuff, because e.g. the esp_vfs_t doesn't match at my side currently. It looks like I still get IDF v3.2 via platformio. :-(

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/me-no-dev/ESPAsyncWebServer/issues/792#issuecomment-645563195, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABATPIC566GHUB24N3IQTE3RXEHKRANCNFSM4OAZ37NQ .

lorol commented 4 years ago

@BlueAndi - I got what you say ... Well I used the git version of the core, I need to note it somewhere.

Less-likely I will go back to fix. Sorry.

BlueAndi commented 4 years ago

With PIO I get more possibilities, as you mention as F1 car and I use them. ;-) I will try it again, if I get IDF 3.3. Anyway thanks for your effort.

lorol commented 4 years ago

@BlueAndi
Try now, but first in esp_littlefs.c un-comment the line:

define CONFIG_LITTLEFS_FOR_IDF_3_2

At least it works now with arduino release core 1.0.4. Let me know if PIO likes it this way :)

BlueAndi commented 4 years ago

Ok, thanks. I will try and report back. Because I am very busy, I may need some more days.

lorol commented 4 years ago

I updated LITTLEFS in GitHub to be with #define CONFIG_LITTLEFS_FOR_IDF_3_2 by default, which allows build on several IDF versions, including the release 1.0.4 and POI on Arduino platform - it works.

zekageri commented 4 years ago

It will be really good if i would be able to upload little fs just like spiffs on platformio vscode ( upload file system image ).

I'am really looking for that littlefs because the spiffs can't handle the 16mb partition on my esp32. I will try it out definietly!

lorol commented 4 years ago

@zekageri Just try for fun:

close vscode
In your PIO installation, find a file, named "mkspiffs_espressif32_arduino" [.exe] :)
Go to: https://github.com/earlephilhower/mklittlefs
Download the mklittlefs[.exe] executable for your OS from releases there
rename your original mkspiffs_espressif32_arduino[.exe] to sometting else like mkspiffs_espressif32_arduino_bak[.exe]
put the mklittlefs[.exe] in same folder where mkspiffs_espressif32_arduino_bak[.exe] is
rename mklittlefs[.exe] to mkspiffs_espressif32_arduino[.exe]
Re-open your vscode project
have data folder prepared
Upload File System Image as usual
Enjoy
Don't forget to restore against SPIFFS
Appreciate if you find the right people to implement it properly in PIO (different command, choice, prompt, py script, config through platformio.ini per project ... etc.)

As last-resort, you can keep both executable files there and write a simple shell or batch script to automate the "swap" hack above.

zekageri commented 4 years ago

i'am trying it right now. I have an error currently wich is :

src/Includes.h:32:16: error: conflicting declaration 'fs::SPIFFSFS LITTLEFS'
 #define SPIFFS LITTLEFS

It looks like LITTLEFS defined somewhere already.

EDIT: if i modify it to #define SPIFFS LITTLEFSFS ( because in LITTLEFS.cpp there is this: LITTLEFSFS LITTLEFS; ) i get an other error wich says :

.pio\build\esp32dev\src\main.cpp.o:(.literal._ZNSt17_Function_handlerIFvP21AsyncWebServerRequestEZL13HTTP_RequestsvEUlS1_E23_E9_M_invokeERKSt9_Any_dataOS1_+0x0): undefined reference toLITTLEFSFS'`

EDIT2 :

Okay nevermind, i renamed all my SPIFFS. to LITTLEFS. The Code compiles now. ^^

EDIT3 :

I managed to upload the littlefs on PIO with the trick you suggested. No problem, the esp booted up and running with the new file system. After that i forget that i need a couple of more things so i grapped these, copyed to the data folder, and uploaded it again. Now it says that:

[E][vfs_api.cpp:22] open(): File system is not mounted
Guru Meditation Error: Core  1 panic'ed (IntegerDivideByZero). Exception was unhandled.

Even if i put this to the setup:

if(!LITTLEFS.begin(FORMAT_LITTLEFS_IF_FAILED)){
        Serial.println("LITTLEFS Mount Failed");
        return;
    }

And even with fresh upload with the plugin from PIO.

EDIT4:

I belive it is because the JPEGDecoder library. It is using the SPIFFS. I will try it out!

JPEGDecoder by Bodmer

EDIT5: Almost everything is working. The first problem was that this lib used the SPIFFS.h. I changed this too to LITTLEFS and it is okay now. But if some file is not present, ( or not existent ) on the file system, i get an exception and a reboot with backtrace:

E (2424) esp_littlefs: Failed to opendir "/Use
[37mrProg.json". Error  (-2)
fs_api.cpp:64] open(): /littlefs/UserProg.json does not existexist      
E (38744) esp_littlefs: Failed to open
7mir "/NotFound.html". Error  (-2)
E (38751) esp_littlefs: Failed to opendir "/Not
Found.html.gz". Error  (-2)
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhan
dled.
Core 1 register dump:
PC      : 0x400d4cd4  PS      : 0x00060830  A0      : 0x800e3768  A1      : 0x3ffd9cb0
A2      : 0x00000000  A3      : 0x3ffc6a08  A4      : 0x3ffdd194  A5      : 0x3ffc6a08
A6      : 0x0000000d  A7      : 0x00000000  A8      : 0x800d4cd1  A9      : 0x3ffd9c90
A10     : 0x3ffd9cc0  A11     : 0x3f4017b4  A12     : 0xbdc8e301  A13     : 0x3ffc6a08
A14     : 0x00000000  A15     : 0x3ffd9ccc  SAR     : 0x00000004  EXCCAUSE: 0
x0000001c
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x                    4
000150d  LCOUNT  : 0xfffffffb  

Backtrace: 0x400d4cd4:0x3ffd9cb0 0x400e3765:0x3ffd9d00 0x400e136d:0x3ffd9d40 0x400e1431:0x3ffd9d80 0x400e1699:0x3ffd9dc0 0x4015408d:0x3ffd9de0 0x40154109:0x3ffd9e20 0x401547b2:0x3ffd9e40 0x400893c1:0x3ffd9e70
  #0  0x400d4cd4:0x3ffd9cb0 in std::_Function_handler<void (AsyncWebServerRequest*), HTTP_Requests()::{lambda(AsyncWebServerRequest*)#1}>::_M_invoke(std::_Any_data 
const&, AsyncWebServerRequest*&&) at src/Setups.h:181
      (inlined by) _M_invoke at c:\users\dr.random\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf\include\c++\5.2.0/functional:1871
  #1  0x400e3765:0x3ffd9d00 in std::function<void (AsyncWebServerRequest*)>::operator()(AsyncWebServerRequest*) const at lib\ESPAsyncWebServer\src/StringArray.h:73 
      (inlined by) AsyncCallbackWebHandler::handleRequest(AsyncWebServerRequest*) 
at lib\ESPAsyncWebServer\src/WebHandlerImpl.h:123
  #2  0x400e136d:0x3ffd9d40 in AsyncWebServerRequest::_parseLine() at lib\ESPAsyncWebServer\src/WebRequest.cpp:797
  #3  0x400e1431:0x3ffd9d80 in AsyncWebServerRequest::_onData(void*, unsigned int) at lib\ESPAsyncWebServer\src/WebRequest.cpp:797
  #4  0x400e1699:0x3ffd9dc0 in std::_Function_handler<void (void*, AsyncClient*, void*, unsigned int), AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*)::{lambda(void*, AsyncClient*, void*, unsigned int)#8}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&, std::_Any_data const&, unsigned int&&) at lib\ESPAsyncWebServer\src/WebRequest.cpp:797
      (inlined by) _M_invoke at c:\users\dr.random\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf\include\c++\5.2.0/functional:1871
  #5  0x4015408d:0x3ffd9de0 in std::function<void (void*, AsyncClient*, void*, unsigned int)>::operator()(void*, AsyncClient*, void*, unsigned int) const at lib\AsyncTCP\src/AsyncTCP.cpp:1039
      (inlined by) AsyncClient::_recv(tcp_pcb*, pbuf*, signed char) at lib\AsyncTCP\src/AsyncTCP.cpp:934
  #6  0x40154109:0x3ffd9e20 in AsyncClient::_s_recv(void*, tcp_pcb*, pbuf*, signed char) at lib\AsyncTCP\src/AsyncTCP.cpp:1039
  #7  0x401547b2:0x3ffd9e40 in _async_service_task(void*) at lib\AsyncTCP\src/AsyncTCP.cpp:1039
      (inlined by) _async_service_task at lib\AsyncTCP\src/AsyncTCP.cpp:197       
  #8  0x400893c1:0x3ffd9e70 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

I will decode it in a min.

_Nah, can't debug now. Looks like PIO monitor filter ~ monitor_filters = esp32_exception_decoder sometimes not working. But it looks like to me that if multiple files does not exist in the system, it is rebooting. If one or two file, no problem. But if 3 or more then reboot._

_This is not necessarily the LITTLEFS problem, i think Async Webserver has this bug when you rapidly requesting something with HTTPGET then it is going to crash.

lorol commented 4 years ago

use SPIFFS.begin() - with default parameters. not .begin(something)

zekageri commented 4 years ago

use SPIFFS.begin() - with default parameters. not .begin(something)

I get this:

if(!LITTLEFS.begin(FORMAT_LITTLEFS_IF_FAILED)){
        Serial.println("LITTLEFS Mount Failed");
        return;
    }

from LITTLEfs examples.

lorol commented 4 years ago

Check the scope of #define where you do a redefinition of SPIFFS to LITTLEFS . i.e. put it to the sketch, not in .h file ... etc Also SPIFFS.h should not be included anywhere in your project, only LITTLEFS.h

zekageri commented 4 years ago

Yeah i know. Some of the libs using SPIFFS.h. Such as JPEGDecoder as i mentioned. But it was just a problem for about a minute, while i renamed every SPIFFS occourences to LITTLEFS. Now, it seems to mee that everything is working. HTTP file upload, page loads, read write, tft images from fs etc.

zekageri commented 4 years ago

Okay thats weird.

The thing is that if i open a web page on the esp, and not exiting from it. ( this webpage is without websockets ) and open an other webpage from an other device, the esp crashes with this exception code:

[W][AsyncTCP.cpp:969] _poll(): rx timeout 4
E (48880) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (48880) task_wdt:  - async_tcp (CPU 0/1)
E (48880) task_wdt: Tasks currently running:
E (48880) task_wdt: CPU 0: IDLE0
E (48880) task_wdt: CPU 1: loopTask
E (48880) task_wdt: Aborting.
abort() was called at PC 0x400fcba4 on core 0

0x4008cd30: invoke_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 155
0x4008cf61: abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 170
0x400fcba4: task_wdt_isr at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/task_wdt.c line 174
0x40176ac7: esp_pm_impl_waiti at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/pm_esp32.c line 492
0x400fe102: esp_vApplicationIdleHook at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/freertos_hooks.c line 63
0x4008abdd: prvIdleTask at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/tasks.c line 3382
0x400893e9: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

I have no files that does not exist, everything is there, like when i used SPIFFS.

This is where the program crashes:

server.on("/", HTTP_GET, [](AsyncWebServerRequest *request) {
    AsyncWebServerResponse* response = request->beginResponse(LITTLEFS, "/Web_App.html", "text/html");
    response->addHeader("Content-Encoding", "gzip");
    request->send(response);
});

This web is asking for three files when it is loaded. But it does not reach the point where it does actually.

zekageri commented 4 years ago

Oh. It does not happen on windows. And if i open it on android, from the browser, it is loading fine. But if i put the icon into my home screen with chrome, and i open it from there, it is crashing. Wtf

EDIT:

It works on MAC, Windows, and IOS. Tried with ### two windows10 pro laptops, an ### Iphone 7s and an ### Iphone 5s and on a ### MAC. Every device is### loading the page fine except ### android.

Tried with two Realme 3 pro and a Huawei p30 lite. All of them causing a crash with the same exception code as mentioned above.

me-no-dev commented 4 years ago

interesting... seems like Android creates some extra request or something. Did you add onNotFound callback to see what other URLs might be requested? Maybe add a print line so you know when the index is requested?

zekageri commented 4 years ago

I have this:

server.onNotFound([](AsyncWebServerRequest *request){
  AsyncWebServerResponse* response = request->beginResponse(LITTLEFS, "/NotFound.html", "text/html");
    response->addHeader("Content-Encoding", "gzip");
    request->send(response);
});

It is sending a page. All of my files are gzipped. Without the .gz extension so IOS can handle it properly. But i will investigate it further.

zekageri commented 4 years ago

I have on the pio ini:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino

upload_protocol = espota
upload_port = 192.168.0.33
;upload_port = COM3
monitor_speed = 115200
build_type = debug
monitor_filters = esp32_exception_decoder

board_build.flash_size = 16MB (128mb)
board_build.flash_mode = dio
board_build.partitions = large_spiffs_16MB.csv
build_flags = -DCORE_DEBUG_LEVEL=5

In the onNotFound callback:

int notfound = 0;

server.onNotFound([](AsyncWebServerRequest *request){
  notfound++;
  Serial.println(notfound);
  AsyncWebServerResponse* response = request->beginResponse(LITTLEFS, "/NotFound.html", "text/html");
  response->addHeader("Content-Encoding", "gzip");
  request->send(response);
});

Serial does not print anything. Just this exception code when crashing:

E (43484) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (43484) task_wdt:  - async_tcp (CPU 0/1)  
E (43484) task_wdt: Tasks currently running:
E (43484) task_wdt: CPU 0: IDLE0
E (43484) task_wdt: CPU 1: loopTask
E (43484) task_wdt: Aborting.
abort() was called at PC 0x400fd28c on core 0

Backtrace: 0x4008cd30:0x3ffbe170 0x4008cf61:0x3ffbe190 0x400fd28c:0x3ffbe1b0 0x40084ec9:0x3ffbe1d0 0x401771af:0x3ffbc1b0 0x400fe7ea:0x3ffbc1d0 0x4008abdd:0x3ffbc1f0 0x400893e9:0x3ffbc210
  #0  0x4008cd30:0x3ffbe170 in invoke_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:707
  #1  0x4008cf61:0x3ffbe190 in abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:707
  #2  0x400fd28c:0x3ffbe1b0 in task_wdt_isr at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/task_wdt.c:252
  #3  0x40084ec9:0x3ffbe1d0 in _xt_lowint1 at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/xtensa_vectors.S:1154
  #4  0x401771af:0x3ffbc1b0 in esp_pm_impl_waiti at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/pm_esp32.c:492
  #5  0x400fe7ea:0x3ffbc1d0 in esp_vApplicationIdleHook at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/freertos_hooks.c:108
  #6  0x4008abdd:0x3ffbc1f0 in prvIdleTask at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/tasks.c:3507
  #7  0x400893e9:0x3ffbc210 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

zekageri commented 4 years ago

I have tried the tweaked version of the async lib by lorol, but that did not helped.

zekageri commented 4 years ago

The code not even reach this point, where the page should be served:

server.on("/", HTTP_GET, [](AsyncWebServerRequest *request) {
  notfound++;
  Serial.print("NOT FOUND CRASH HERE: ");
  Serial.println(notfound);
    AsyncWebServerResponse* response = request->beginResponse(LITTLEFS, "/Web_App.html", "text/html");
    response->addHeader("Content-Encoding", "gzip");
    request->send(response);
});

zekageri commented 4 years ago

Oh god. Sorry i was dumb AF. I'am using modbus RTU on the Serial and when i commented out the modbus init, i don't Serial.begin(); therefore i don't get any print. The code reach the root point, but when serving, it is crashing.

Zero not found callback happens at that time. Just some rx timeout 4

lorol commented 4 years ago

@zekageri Do the things work as expected now on your side?

zekageri commented 4 years ago

@zekageri Do the things work as expected now on your side?

Nop. The ESP still crashes if i open the webpage from my phone. Interestingly, now i tried with some other phones and PCs, everything works except with two phones, two Realme 3 pro. They are crashing the esp.

One is mine and the other is my girlfriend's phone. We use our phones for completelly different things. We tried it with Chrome and Firefox, but both cause crash.

zekageri commented 4 years ago

Currently trying to debug the problem. I plugged my phone into the laptop, with usb debugging. Inspecting it with Edge devtools on the laptop. It is crashing when the esp wants to serve a "largeish" file. Its a JS file and its only 200kb.

Yes. It is crashing whenever i want to load a page where there is at least one file wich is bigger then 200kb.

I reduced the file to exactly 200kb. It has more success in the loading process but still not 100%. Like 5/2 sucessfull load happening now.

Tried to copy everything into one file. 250kb file, one request, still crashing. So the problem is really with the size.

lorol commented 4 years ago

I don't have an Android phone handy but I uploaded and downloaded back a 550kb file by PC Mozilla browser - with a File Editor as part of ESPAsyncWebServer (my tweak and the SmartSwitch application in my clone) it worked well with LITTLEFS.

zekageri commented 4 years ago

It's still crashing. If i want to send a file wich contains some javascript ( jquery, and font awesome svg (the whole file is 241kb) ) it's just crashing. I dont know. Maybe my 0th core is overloaded and i don't have the memory to send it?

I'am gzipping it with 7-zip and removing the gz extension like on any other file i have and sending like this:

server.on("/Web_App_Big_JS.js", HTTP_GET, [](AsyncWebServerRequest *request) {
    AsyncWebServerResponse* response = request->beginResponse(LITTLEFS, "/Web_App_Big_JS.js", "text/javascript");
    response->addHeader("Content-Encoding", "gzip");
    request->send(response);
});

zekageri commented 4 years ago

Maybe the problem is with the partition. Maybe the async callback can't delay that much time inside it while serving the file. I've got a board_build.partitions = large_spiffs_16MB.csv so the flash is pretty big. I don't have that with SPIFFS because it can't handle that partition table.

BlueAndi commented 4 years ago

Appreciate if you find the right people to implement it properly in PIO (different command, choice, prompt, py script, config through platformio.ini per project ... etc.)

Add to platformio.ini: extra_scripts = replace_fs.py

Add replace_fs.py to project root directory (where platformio.ini is located):

Import("env")
print("Replace MKSPIFFSTOOL with mklittlefs.exe")
env.Replace (MKSPIFFSTOOL = "mklittlefs.exe")

Add mklittlefs.exe to project root directory as well.

lorol commented 4 years ago

I will add your notes to README

BlueAndi commented 4 years ago

@lorol I got it running on my esp32: https://github.com/BlueAndi/esp-rgb-led-matrix/tree/LittleFS Did not update SPIFFSEditor.cpp right now, but will in the next step.

Edit: Your updated ESPAsyncWebServer expects the following additional files, which are missing in the repo.

acefull.js
ace.js
ace.ico

lorol commented 4 years ago

Needed stuff is in gzipped files in /data folder. Let me know which part asks for missing ones. see https://github.com/lorol/ESPAsyncWebServer/blob/master/src/edit.htm and https://github.com/lorol/ESPAsyncWebServer/tree/master/examples/SmartSwitch/data

BlueAndi commented 4 years ago

I rolled back yesterday ;-) Updated the editor now by my own (not pushed right now) to show all files in the subdirectories again.

Implemented the following routine (esp32 only), if you are interesting:

String SPIFFSEditor::getFilesRecursive(const String& path, bool& isFirst)
{
  String result;
  File dir = _fs.open(path);
  File entry = dir.openNextFile();

  while(entry){
    if (true == isExcluded(_fs, entry.name())) {
      /* Nothing to do */
    } else if ((true == entry.isDirectory()) &&
               (0 != strcmp("/", entry.name()))) {
      result += getFilesRecursive(entry.name(), isFirst);
    } else if (false == entry.isDirectory()) {
      if (true == isFirst) {
        isFirst = false;
      } else {
        result += ",";
      }
      result += "{\"type\":\"";
      result += "file";
      result += "\",\"name\":\"";
      result += String(entry.name());
      result += "\",\"size\":";
      result += String(entry.size());
      result += "}";
    }

    entry.close();
    entry = dir.openNextFile();
  }

  dir.close();

  return result;
}

lorol commented 4 years ago

@BlueAndi thanks for the code shown. I did the entering to folder by click over directory name, identical for deleting. (right click) with silent tree removal on last file.

lorol commented 4 years ago

https://github.com/lorol/arduino-esp32fatfs-plugin You can do FAT partition replacement tests on Arduino, too. Just get this tool, select a part. scheme w/fat and see the examples.

The espota works too, but needs a modification of esp32-core library named "Update", see in the extra folder in above link

Note, when you try same board with different partition schemes and/or filesystems one after the other tests, you need to flash 2-3 times the program flash and data partition both, and at least one of them by serial port not ota to make sure everything is good to go with the new filesystem.

BlueAndi commented 4 years ago

@lorol Did you make some measurement regardin read/write speed with LittleFS compared to SPIFFS or FAT?

atanisoft commented 4 years ago

@BlueAndi https://github.com/RIOT-OS/RIOT/pull/8316 (similar comparison in another project).

and from esp_littlefs itself: https://github.com/joltwallet/esp_littlefs#performance

lorol commented 4 years ago

@BlueAndi https://github.com/lorol/LITTLEFS#differences-with-spiffs - see the table these are just from generic sketch. ~~For some reason the reading of LittleFS of this sketch setup shows slower than SPIFFS, could be the measurement criteria is not correctly set or ... on esp32 platform this is something to expect, or who knows :) This RIOT-OS thing looks like covers different HW, so the architecture may bring differences.~~ Edit: If you change in esp_littlefs.c the cache to 512: #define CONFIG_LITTLEFS_CACHE_SIZE 512 then the read speed of example sketch gets 2 x times faster, so it is indeed faster than SPIFFS

lorol commented 4 years ago

BTW, i combined all 3 uploaders to a single one with drop-down choice. The mk..fs binaries are collected there, too. See: https://github.com/lorol/arduino-esp32fs-plugin

lorol commented 4 years ago

Thank you for the feedback. I submitted a request LITTLEFS for esp32 to be reviewed for Arduino Library Manager https://github.com/arduino/Arduino/issues/10719

Closing this.

BlueAndi commented 4 years ago

@lorol Thanks for sharing. Curious about the IDF integration.

gnalbandian commented 4 years ago

@BlueAndi are you planning on releasing the tweaked version of spiffseditor that works with LittleFS?

BlueAndi commented 4 years ago

You will find the version of @lorol here: https://github.com/lorol/ESPAsyncWebServer Any my one here: https://github.com/BlueAndi/ESPAsyncWebServer/tree/LittleFS

me-no-dev / ESPAsyncWebServer

LittleFS and FatFS on ESP32, feedback needed #792

include

define SPIFFS LITTLEFS

include

define CONFIG_LITTLEFS_FOR_IDF_3_2

EDIT: if i modify it to #define SPIFFS LITTLEFSFS ( because in LITTLEFS.cpp there is this: LITTLEFSFS LITTLEFS; ) i get an other error wich says :

EDIT2 :

EDIT3 :

Even if i put this to the setup:

EDIT4:

EDIT5: Almost everything is working. The first problem was that this lib used the SPIFFS.h. I changed this too to LITTLEFS and it is okay now. But if some file is not present, ( or not existent ) on the file system, i get an exception and a reboot with backtrace:

This is where the program crashes:

EDIT:

Tried with two Realme 3 pro and a Huawei p30 lite. All of them causing a crash with the same exception code as mentioned above.

I have this:

I have on the pio ini:

In the onNotFound callback:

The code not even reach this point, where the page should be served:

Zero not found callback happens at that time. Just some rx timeout 4