search

fredlcore / BSB-LAN

LAN/WiFi interface for Boiler-System-Bus (BSB) and Local Process Bus (LPB) and Punkt-zu-Punkt Schnittstelle (PPS) with a Siemens® controller used by Elco®, Brötje® and similar heating systems
220 stars 83 forks source link

Limit SPIFFS use (ESP32) #557

Closed DE-cr closed 1 year ago

DE-cr commented 1 year ago

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/spiffs.html

"SPIFFS is able to reliably utilize only around 75% of assigned partition space."

DE-cr commented 1 year ago

Although the official sources say that only 75 % of SPIFFS can be reliably used, I'll keep the old #define MINIMUM_FREE_SPACE_ON_SD 10000 instead of 1.5 MB * 0,25 == 393216 for my system.

Reason: even 1.5 MB isn't plenty for data logging (when you don't have an SD card in your system), and having that reduced to <1.2 MB for "reliable use" only makes matters worse in that regard. Also, in my "testing" (extremely limited, just one test case!), using more than 75 % of SPIFFS did work:

With 631516 bytes of SPIFFS space available, I tried to use /DK1 (c.f. https://github.com/fredlcore/BSB-LAN/pull/542#issuecomment-1471448906) on a datalog.txt of 740704 bytes, only to receive the error message "Not enough space on device" with the 25 % reserve requirement.

After reducing the reserve to the original 10000 bytes (< 1.6 %), /DK1 did work - albeit it took two attempts, and it did take longer than expected:

$ time wget -qO- bsb-lan/DK1
.......... .......... ...
Error writing datalog

real    0m21,897s
user    0m0,001s
sys 0m0,009s
$ time wget -qO- bsb-lan/DK1
.......... .......... .....
Success

real    0m42,337s
user    0m0,001s
sys 0m0,008s

For completeness: After /DK1, my datalog.txt was 263379 bytes, and free SPIFFS was 1111679. With a 25 % reserve, "not enough space" was correct: (datalog:263379 + reserve:393216 == 656595) > spiffs:631516.

It's up to you to decide whether to merge this PR or not.

fredlcore commented 1 year ago

The question is whether a switch to LittleFS would make more sense?

DE-cr commented 1 year ago

Having just read about LittleFS, I'd say: In the long term, using it seems to be the way to go. Right now, it seems easier to use SPIFFS instead, and SPIFFS seems to be "good enough" in most cases.

fredlcore commented 1 year ago

I haven't seen the need for it so far as well, but if it means that the already limited space on a ESP32 without SD card is further reduced by 25%, it might be worth looking at it more closely. I read somewhere that the library calls are mostly compatible, so it might be as easy as changing the included library, but I'll see when I find time to look at it more closely...

fredlcore commented 1 year ago

It actually is just a drop-in replacement. Tested it on my ESP32 Olimex with its internal memory and logging, displaying and removing log files work fine. So I'll keep the minimum at 10000 bytes for now.

fredlcore commented 1 year ago

Since you did a few tests regarding speed, I wonder if LittleFS will produce noticeable gains as well...

DE-cr commented 1 year ago

First simple test to compare LittleFS to SPIFFS, in a sketch doing just that: grafik ...and yes, it's really as simple as #define SPIFFS LittleFS and #including a different header file - once you've added the esp_littlefs library in your Arduino IDE. One more timing comparison: formatting took 5.786 s with LittleFS vs. 26.536 s with SPIFFS.

fredlcore commented 1 year ago

Shouldn't be necessary to add LittleFS as it is part of the ESP core v2. If that's the library you've added, that's the predecessor and it's outdated: https://github.com/lorol/LITTLEFS

DE-cr commented 1 year ago

OK, I've just searched for "littlefs" in the library manager of my Arduino 1.8.19 IDE: nothing found. Maybe I didn't install anything extra?

DE-cr commented 1 year ago

Using a different computer, I can confirm that I didn't have to install LittleFS. :)

Some more timings:

=> In my tests, LittleFS has always been considerably faster than SPIFFS.

Also, with LittleFS files up to 1400 KB could be written, as opposed to just 1200 KB with SPIFFS. => https://github.com/fredlcore/BSB-LAN/pull/557#issuecomment-1475251447

Considering that LittleFS doesn't require an extra installation and that the code change would be trivial, you should definitely consider changing bsb-lan usage of SPIFFS to LittleFS.

I have not done any reliability testing for LittleFS, though! However, from what I can see on the net, it seems to be trustworthy.

Do you want me to prepare a pull request? :)

fredlcore commented 1 year ago

I've already included in the master repo when I closed this PR ;)...

fredlcore commented 1 year ago

But thanks for testing, the advantages are more than obvious now...