joltwallet / esp_littlefs

LittleFS port for ESP-IDF
MIT License
254 stars 95 forks source link

What is the correct behaviour when the filesystem is 100% full #183

Closed adokitkat closed 4 months ago

adokitkat commented 4 months ago

Hello. I am asking this in regard of a forum post.

The issue is described here: https://esp32.com/viewtopic.php?f=13&t=39197&p=130708

What should happen if the partition is full? The user mentions the error is coming directly from LittleFS internals - " I expected a graceful failure of fopen() or fprintf() but I get a DivideByZero exception from lfs_alloc in the bowels of the (LittleFS) library from fclose()".

However since he's using ESP32, I figured I'll try to ask here first. Could you please try to point me what is the real issue etc.? Should I open an issue directly in littlefs repo? Many thanks.

BrianPugh commented 4 months ago

I'm following up directly in the thread; will post the summary here once a conclusion has been reached.

huming2207 commented 4 months ago

Hey @adokitkat and @BrianPugh ,

We've also experienced the same issue last week. Basically we were running LittleFS on a ESP32 partition on SPI NOR flash (not SD card). I remembered we have the exact 4096 bytes free space left but we couldn't write any more stuff. If we do fwrite() follow by fflush() it will then end up in the DivideByZero exception. We also couldn't delete anything. If we call remove() or unlink() it will also crash because of DivideByZero. But I doubt it might be because we created/formatted that partition on some older LittleFS libraries a year ago, so it might cause some corruption. We have re-formatted the partition with the latest LittleFS and esp-littlefs wrapper library and it seems working fine for now. But of course we haven't written that partition to full yet, we can't tell whether that is a bug on LittleFS or we misused the LittleFS across different versions. We will leave it running for a while and see what will happen later.

Meanwhile @adokitkat can you confirm which LittleFS/esp-littlefs library did you use use when you created that partition? How much space do you left right now?

Regards, Jackson

huming2207 commented 4 months ago

Here we go again:

This time we managed to catch the stack trace. We are using ESP-IDF v5.1.2 + esp-littlefs v1.14.2 (with LittleFS v2.9.0), on SPI NOR flash partition:

Guru Meditation Error: Core  0 panic'ed (IntegerDivideByZero). Exception was unhandled.

Core  0 register dump:
PC      : 0x42020a09  PS      : 0x00060930  A0      : 0x82020c15  A1      : 0x3fca5f50  
0x42020a09: lfs_alloc at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/littlefs/lfs.c:689 (discriminator 3)

A2      : 0x00000000  A3      : 0x00001241  A4      : 0x3fcb6254  A5      : 0x3fcbebec  
A6      : 0x00000000  A7      : 0x00000800  A8      : 0x3c0bfc34  A9      : 0x3fca5f30  
A10     : 0x00004265  A11     : 0x3c0a04c8  A12     : 0x00000008  A13     : 0x00000800  
A14     : 0x00000800  A15     : 0x00000800  SAR     : 0x00000020  EXCCAUSE: 0x00000006  
EXCVADDR: 0x00000000  LBEG    : 0x40056f5c  LEND    : 0x40056f72  LCOUNT  : 0xffffffff  
0x40056f5c: memcpy in ROM
0x40056f72: memcpy in ROM

Backtrace: 0x42020a06:0x3fca5f50 0x42020c12:0x3fca5f80 0x42020e0d:0x3fca5fe0 0x420210c5:0x3fca6020 0x420239f5:0x3fca6050 0x4201e986:0x3fca6070 0x42009b5b:0x3fca60a0 0x4207a675:0x3fca60c0 0x42078da1:0x3fca60e0 0x42078e26:0x3fca6100 0x42078e75:0x3fca6130 0x4201c77e:0x3fca6150 0x4201d6df:0x3fca6170 0x4201d897:0x3fca6630 0x4201d8bd:0x3fca6660 0x4200ce54:0x3fca6690 0x4208d6cb:0x3fcaaea0 0x40384eb9:0x3fcaaed0
0x42020a06: lfs_alloc at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/littlefs/lfs.c:689 (discriminator 3)
0x42020c12: lfs_ctz_extend at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/littlefs/lfs.c:2892
0x42020e0d: lfs_file_flushedwrite at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/littlefs/lfs.c:3557
0x420210c5: lfs_file_write_ at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/littlefs/lfs.c:3639
0x420239f5: lfs_file_write at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/littlefs/lfs.c:6134
0x4201e986: vfs_littlefs_write at /home/whoever/Projects/weirdstuff/meh/components/esp_littlefs/src/esp_littlefs.c:1509
0x42009b5b: esp_vfs_write at /home/whoever/esp/esp-idf/components/vfs/vfs.c:445 (discriminator 3)
0x4207a675: __swrite at /builds/idf/crosstool-NG/.build/xtensa-esp32s3-elf/src/newlib/newlib/libc/stdio/stdio.c:94
0x42078da1: __sflush_r at /builds/idf/crosstool-NG/.build/xtensa-esp32s3-elf/src/newlib/newlib/libc/stdio/fflush.c:224
0x42078e26: _fflush_r at /builds/idf/crosstool-NG/.build/xtensa-esp32s3-elf/src/newlib/newlib/libc/stdio/fflush.c:278
0x42078e75: fflush at /builds/idf/crosstool-NG/.build/xtensa-esp32s3-elf/src/newlib/newlib/libc/stdio/fflush.c:291
0x4201c77e: sq_esp_disk_io::flush() at /home/whoever/Projects/weirdstuff/meh/components/sq/sq_esp_disk_io.cpp:176
0x4201d6df: queue::append(unsigned char, unsigned char*, unsigned int, unsigned long, sq::record_header*, bool) at /home/whoever/Projects/weirdstuff/meh/components/sq/queue.cpp:125 (discriminator 2)
0x4201d897: queue::append(unsigned char, unsigned char*, unsigned int, sq::record_header*, bool) at /home/whoever/Projects/weirdstuff/meh/components/sq/queue.cpp:60
0x4201d8bd: queue::append_boot() at /home/whoever/Projects/weirdstuff/meh/components/sq/queue.cpp:244
0x4200ce54: app_main at /home/whoever/Projects/weirdstuff/meh/main/main.cpp:275
0x4208d6cb: main_task at /home/whoever/esp/esp-idf/components/freertos/app_startup.c:208 (discriminator 13)
0x40384eb9: vPortTaskWrapper at /home/whoever/esp/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162
huming2207 commented 4 months ago

Looks like the block_count become 0 somehow:

https://github.com/littlefs-project/littlefs/blob/4dd30c1b8f1b416633fe63a338ede8934b6449a9/lfs.c#L689-L691

I had a quick look it seems like the superblock's block_count somehow been set to 0? So then the block_count in lfs_cfg was also set to 0 later.

@geky do you have any idea? 😅

BrianPugh commented 4 months ago

@huming2207 do you set lfs_config.block_count to zero (autodetect from superblock)? Because if so, then this one error log is a simple oversight (it should be lfs->block_count instead of lfs->cfg->block_count). If you can confirm this is the case, I'll open up a PR in upstream littlefs.

huming2207 commented 4 months ago

do you set lfs_config.block_count to zero (autodetect from superblock)? Because if so, then this one error log is a simple oversight (it should be lfs->block_count instead of lfs->cfg->block_count). If you can confirm this is the case, I'll open up a PR in upstream littlefs.

@BrianPugh I'm checking with my colleagues. I will come back to you later.

BrianPugh commented 4 months ago

sounds, good; in the meantime i'll make a PR because it definitely looks like a bug, regardless.

huming2207 commented 4 months ago

@BrianPugh We can confirm that is the issue 😅 By changing the lfs->cfg->block_size to lfs->block_size will fix the crash.

adokitkat commented 4 months ago

Thank you both a lot!

BrianPugh commented 4 months ago

fixed in core littlefs v2.9.2, and subsequently in esp_littlefs v1.14.4