SmingHub / Sming

Sming - powerful open source framework simplifying the creation of embedded C++ applications.
https://sming.readthedocs.io
GNU Lesser General Public License v3.0
1.45k stars 349 forks source link

SPIFFS crash on api_spiffs_erase (example provided) #1660

Open kmihaylov opened 5 years ago

kmihaylov commented 5 years ago

Hello. Finally I found when my FS is crashing. It is some time after data has been written to it, that's why it was a little bit hard to figure it out when exactly the bug appears.

I found that while there is no need for api_spiffs_erase, the consistency of the FS is OK, the device can be rebooted w/o problems (data loss, broken FS).

On the first appearance of "api_spiffs_erase", however, the FS is immediately broken.

I made a simple program where one can write some data to the FS. Issue command save 10 where the integer is how many times the fileSetContent will be called. The sample text is in the cpp file and can be changed. I set it to 10 bytes for convenience (idk how SPIFFS reserves bytes for the filename and etc.).

You have to try with some random repetition values, 100 is good to begin with. When you figure out where "api_spiffs_erase" is shown you can decrease the argument of saveto this position -1.

I got "api_spiffs_erase" on the 63rd write (630 bytes of data excluding what bytes were reserved for file name, spiffs, etc).

If I try to repeat the file save 62 times, the FS works OK, the device can be safely rebooted w/o data loss. After the reboot only one save operation is needed to reach "api_spiffs_erase" and the FS gets immediately broken. On the next reboot it is formatted by the firmware.

Example code is provided here: https://github.com/kmihaylov/Sming_fsTest.git

Tested with commit 5300c0b (current develop)

/opt/esp-open-sdk/esptool/esptool.py -p /dev/ttyUSB0 -b 115200 write_flash -ff 40m -fm qio -fs 32m 0x00000 out/firmware/rboot.bin 0x02000 out/firmware/rom0.bin 0x100000 out/firmware/spiff_rom.bin
WARNING: Flash size arguments in megabits like '32m' are deprecated.
Please use the equivalent size '4MB'.
Megabit arguments may be removed in a future release.
esptool.py v2.6-beta1
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: bc:dd:c2:91:cf:b1
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Compressed 2032 bytes to 1429...
Wrote 2032 bytes (1429 compressed) at 0x00000000 in 0.1 seconds (effective 123.9 kbit/s)...
Hash of data verified.
Compressed 275280 bytes to 196722...
Wrote 275280 bytes (196722 compressed) at 0x00002000 in 17.5 seconds (effective 126.1 kbit/s)...
Hash of data verified.
Compressed 65536 bytes to 107...
Wrote 65536 bytes (107 compressed) at 0x00100000 in 0.0 seconds (effective 38119.2 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
python -m serial.tools.miniterm /dev/ttyUSB0 115200 
--- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
--- DTR inactive ---
--- RTS inactive ---
r$␀$ܟ|␀�$�|␂␌␌␄�␄d�␄b|��␃�2�r�b�␌#��on�lNo���␌"␜p��l{l{lp�n�␐␃␌␄�␄l␄␄␌␄␌␌#␌o�|␃$�␄��␌#��on�␂l��␀$`␂�␛␒on␄l ␃␏␃os���n␄␌␒� ␂p�o�␐␂␌␄{�ܜ��␌␄␌␄c␄o�|␂����␌#��nN�␂␌␄l ␃�␒2nol�l ␃␏␃os���n␄␌b�� ␂␏rےo␌␄#��`␃rd�#�on���p|␄�b␜␀lB�␓op`␒#�2c�#��N␀��n�b�N#p␡�n␏l␜�o�␂n����␟␀␌␄�␌l$pc�� ␃␜�o�␂ol$n��#␄␌␌d�$l␏l␜�o�␂o�␒���b␌␌␄��␄lld~sl�b�no��d�␒non>␀�␓o܄�N�␃o␎$r$␄�on���N��|~�o␌�b␏l��␃�␛�o��r��o|�␌l␌ldl ␂��{�l�l␒�␄␌␄l ␃��r�d�␜�␛�␌␄␌$`␂��r�$␌��␀�␟␀␌dl ␂��␛�n�␃N␎$rdN␑-�ver275956 Commandhandlercommand status registered
276383 Commandhandlercommand echo registered
276693 Commandhandlercommand help registered
277128 Commandhandlercommand debugon registered
281304 Commandhandlercommand debugoff registered
285572 Commandhandlercommand command registered
Welcome to the Stream Command executor
292990 trying to mount spiffs at 0x00100000, length 65536
298026 fs.start: size:64 Kb, offset:0x100000

302024 First init file system
304608 sect_first: 100, sect_last: 10f

333003 6%
360833 12%
385806 18%
408859 25%
433852 31%
458776 37%
483740 43%
508669 50%
531721 56%
556627 62%
579633 68%
602679 75%
627674 81%
652644 87%
676649 93%
699670 100%
699874 formatted
701453 mount res: 0

706169 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
save 62

Full cmd is: save 62
Cmd1: save, cmd2: 62
6782446 Bytes written to fs: 10
6790006 Bytes written to fs: 20
6797567 Bytes written to fs: 30
6805116 Bytes written to fs: 40
6812667 Bytes written to fs: 50
6820200 Bytes written to fs: 60
6827732 Bytes written to fs: 70
6835249 Bytes written to fs: 80
6842771 Bytes written to fs: 90
6850274 Bytes written to fs: 100
6857774 Bytes written to fs: 110
6865261 Bytes written to fs: 120
6872752 Bytes written to fs: 130
6880222 Bytes written to fs: 140
6887684 Bytes written to fs: 150
6895146 Bytes written to fs: 160
6902660 Bytes written to fs: 170
6910205 Bytes written to fs: 180
6917749 Bytes written to fs: 190
6925280 Bytes written to fs: 200
6932808 Bytes written to fs: 210
6940322 Bytes written to fs: 220
6947828 Bytes written to fs: 230
6955321 Bytes written to fs: 240
6962820 Bytes written to fs: 250
6970297 Bytes written to fs: 260
6977769 Bytes written to fs: 270
6985232 Bytes written to fs: 280
6992692 Bytes written to fs: 290
7000132 Bytes written to fs: 300
7007628 Bytes written to fs: 310
7015012 Bytes written to fs: 320
7022632 Bytes written to fs: 330
7030144 Bytes written to fs: 340
7037635 Bytes written to fs: 350
7045125 Bytes written to fs: 360
7052602 Bytes written to fs: 370
7060087 Bytes written to fs: 380
7067544 Bytes written to fs: 390
7075001 Bytes written to fs: 400
7082442 Bytes written to fs: 410
7089877 Bytes written to fs: 420
7097296 Bytes written to fs: 430
7104715 Bytes written to fs: 440
7112122 Bytes written to fs: 450
7119526 Bytes written to fs: 460
7126913 Bytes written to fs: 470
7134340 Bytes written to fs: 480
7141796 Bytes written to fs: 490
7149252 Bytes written to fs: 500
7156688 Bytes written to fs: 510
7164122 Bytes written to fs: 520
7171544 Bytes written to fs: 530
7178966 Bytes written to fs: 540
7186368 Bytes written to fs: 550
7193766 Bytes written to fs: 560
7201155 Bytes written to fs: 570
7208537 Bytes written to fs: 580
7215917 Bytes written to fs: 590
7223281 Bytes written to fs: 600
7230636 Bytes written to fs: 610
7238041 Bytes written to fs: 620
--- RTS active ---
--- RTS inactive ---
{l␀$��|␀�$�|␃␄␌␄�␄d�␄b|����␛�{�"�␄b��no�$on��#␄c␜p�c$`␂r$p�o�␐␂␌␌�␌$␌␄␌␌␄␌B␌n�|␃$�␄��p␄��on�␂l��␀$ ␂�␛␒nN␌l ␂␏␂nr����␂␌␄␛�`␃p�o�␐␂␌␄r�����␌␌␄␌#␌N�|␃���␄␌"��on�␂␌␄l`␃�␒2nol�l`␃␎␃or���N␌␌"��`␂␎{��n␄␌b�� ␂sl�b�no߄���c�|�␌$b�2np ␛b�␒"�b��o␀���␂␛Nobp~�o␎$␜�n�␃o�o~␒"␌␄␌�␌$lpb��`␂␜�n�␃n$lo���␌␌␄l�ll␎$␜�n�␃n�␛�ۂB␌␄␌�␌$dl␡rd�#�on��$�␒oNo␞␀�␛o܄�n�␃o␎$r$␌�oo��|�␂��o�␂␄�#␎$�œ�␒�N�{��o|�␄l␄ldl ␂�œo�$��: 0079599 Commandhandlercommand status registered
80022 Commandhandlercommand echo registered
80328 Commandhandlercommand help registered
80640 Commandhandlercommand debugon registered
84597 Commandhandlercommand debugoff registered
88778 Commandhandlercommand command registered
Welcome to the Stream Command executor
96109 trying to mount spiffs at 0x00100000, length 65536
101058 fs.start: size:64 Kb, offset:0x100000

106535 mount res: 0

106895 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
save 1

Full cmd is: save 1
Cmd1: save, cmd2: 1
4205519 api_spiffs_erase
4230420 api_spiffs_erase
4261372 Bytes written to fs: 10
--- RTS active ---
--- RTS inactive ---
{l␀l���␃�l�|␂␌␄␌�␄l�␌b|��␂�␛�r�b�␄c��on�lNo���␌"␜p�l{lrdp�o�␐␃␄␌�␌l␄␌␄␄␌␄c␌o��Nl�␄��␄c��nN�␂l��␀l ␂�2␛on␄l`␃␎␃N{���n␄␌␛�� ␃p�o�␐␃␄␌r�ܜ��␌␄␌␄c␌o�|␃���␄␌b��No�␃␄␌$ ␂�␒␛nol�l`␃␎␃N{���n␄␄#��`␃␎{��n␄␌b��␎l{l�#�oN���p|␌�b␜␀lb�␛Np ␛b�␒"�b��o␀��n�"�Ncp~�n␏l␜�o�␂o�o~␒b␌␄␌�␜�␌$dpc�� ␃␜�o�␂ol$n���␌␄␌$�l$␎$␜�n�␃N�␛�ۂ"␌␄␌��␄l$l~r$�b�No��l�␒noo␞␀�␛o܌�n�␂n␎l{l␌�no�#�o܄�n�␂␌�#␏d��␃�␛�n��r��n|�␌$␌$l$`␂�œo�$��: 0079745 Commandhandlercommand status registered
80169 Commandhandlercommand echo registered
80475 Commandhandlercommand help registered
80786 Commandhandlercommand debugon registered
84743 Commandhandlercommand debugoff registered
88924 Commandhandlercommand command registered
Welcome to the Stream Command executor
96256 trying to mount spiffs at 0x00100000, length 65536
101204 fs.start: size:64 Kb, offset:0x100000

105203 First init file system
107787 sect_first: 100, sect_last: 10f

136173 6%
163947 12%
188700 18%
211540 25%
236320 31%
261114 37%
285814 43%
310556 50%
333560 56%
358432 62%
381405 68%
404403 75%
428366 81%
453333 87%
476369 93%
501263 100%
501466 formatted
503041 mount res: 0

507755 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
mikee47 commented 5 years ago

@kmihaylov You've highlighted the problem very accurately! The logic in spiffs_mount_internal() (spiffs_sming.c line 114) is too simplistic -

  uint32_t dat;
  bool writeFirst = false;
  flashmem_read(&dat, cfg->phys_addr, 4);
  //debugf("%X", dat);

  if (dat == UINT32_MAX)
  {
      debugf("First init file system");
      spiffs_format_internal(cfg);
      writeFirst = true;
  }

The volume gets wiped if SPIFFS sector #0 is in the erased state when the device reboots, which is a perfectly legitimate state for SPIFFS to be in.

Here's a better way:

// Call SPIFFS_mount() with appropriate parameters
int res = do_mount();
// Check result
if(res < 0) {
    // Mount failed, format the volume then retry
    format();
    res = do_mount();
}
// Final result of mount operation
return res;

There's also a volume checking function in SPIFFS, which is time-consuming but could be integrated like this:

int res = do_mount();
if(res < 0) {
    // Mount failed, perform a volume check
    check_fs();
    res = do_mount();
    if(res < 0) {
        format();
        res = do_mount();
    }
}
return res; // Final result of mount operation

You can find some example code for the above here

kmihaylov commented 5 years ago

I did as you suggested:

https://github.com/kmihaylov/Sming/blob/fix/spiffs_mount_prevent_format/Sming/Services/SpifFS/spiffs_sming.c

Now it successfully mounts the fs every time (even if I flashed the device with DISABLE_SPIFFS = 1). The files are written to the FS, but I'm curious how i works if it wasn't inited. Also the problem of unnecessary inited FS disappeared and the files seem to contain the data they should contain.

SPIFFS magically inits the FS on the fly or something else happens that I don't understand yet.

mikee47 commented 5 years ago

Happy that's working for you! The DISABLE_SPIFFS only controls whether an image is built and uploaded during flash.

SPIFFS magically inits the FS on the fly

Pretty much! It doesn't erase sectors unless it needs the space. Have you tested the check_fs() part yet? If you wanted to do some stress testing, try filling the flash with random data first. That should fail the first mount operation and force it to do a check_fs().

kmihaylov commented 5 years ago

It mounts the FS despite it was prefilled with random data.

/opt/esp-open-sdk/esptool/esptool.py -p /dev/ttyUSB0 -b 115200 write_flash -ff 40m -fm qio -fs 32m \
0x00000 out/firmware/rboot.bin \
0x02000 out/firmware/rom0.bin \
0x100000 random.bin && python -m serial.tools.miniterm /dev/ttyUSB0 115200

The mount() function is called 10 seconds after boot to ensure that the debug messages refer to the first mount of the random-data space (I had fear that it formats the FS too quickly).

fileSetContent however now returns open errno -10001.

FS is unusable.


Flashing zeros to the SPIFFS area enables successful fileSetContent() operation, but the FS check returns "-10024"

$ /opt/esp-open-sdk/esptool/esptool.py -p /dev/ttyUSB0 -b 115200 write_flash -ff 40m -fm qio -fs 32m 0x100000 zero.bin 
WARNING: Flash size arguments in megabits like '32m' are deprecated.
Please use the equivalent size '4MB'.
Megabit arguments may be removed in a future release.
esptool.py v2.6-beta1
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: bc:dd:c2:91:cf:b1
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Compressed 65536 bytes to 84...
Wrote 65536 bytes (84 compressed) at 0x00100000 in 0.0 seconds (effective 44188.1 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
$ python -m serial.tools.miniterm /dev/ttyUSB0 115200
--- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
--- DTR inactive ---
--- RTS inactive ---
{l␀$ܞ|␀�l�|␂␌␄␌�␄l�␌#|����␒�r�B�␌#�p~�o�lon���␌B␜p��cl ␃{lp�N�␐␂␌␄�␌$␌␄␌␄␌␌B␌o��ol�␄��␄b��oN�␃d��␀$`␂�2␒oN␌$`␃␎␃or�ےn␄␌2�� ␂p�n�␐␂␌␄{�ܜ��␌␄␌␄b␄N�|␂����␌"��no�␃␌␄l ␂�␒␛ond�␜{l␏␂n{���o␌␌b��`␃␎rےn␄␌B�� ␂r$�#�no���p|␄�|�␄lB�|Br␃N��␓B�#��o␀��␛␒no"p␡�n␎$␜�N�␂n�o~␒#␄␌␄�␄l$pc��`␂␜�o�␂nl$n��c␄␌␌$�$l␎$␜�o�␂N�␒����>␀␌␄��␌$l$~r$�#�no��$�␛Nno␞␀�␒n���N�␂n␎$rd␌�on�#�n��|␡�o␄�#␏l��␃�␓�o�r��o|�␌l␄$ll ␂��r�l��: 0079839 Commandhandlercommand status registered
80249 Commandhandlercommand echo registered
80548 Commandhandlercommand help registered
80850 Commandhandlercommand debugon registered
84834 Commandhandlercommand debugoff registered
89011 Commandhandlercommand command registered
Welcome to the Stream Command executor
96380 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
10101001 FS check res: -10024
10101225 trying to mount spiffs at 0x00100000, length 65536
10101420 fs.start: size:64 Kb, offset:0x100000

10103543 mount res: 0

save 1

Full cmd is: save 1
Cmd1: save, cmd2: 1
27644332 api_spiffs_erase
27688299 api_spiffs_erase
27712270 api_spiffs_erase
27734290 api_spiffs_erase
27757976 api_spiffs_erase
27779983 api_spiffs_erase
27805229 api_spiffs_erase
27825398 api_spiffs_erase
27847614 Bytes written to fs: 10
--- RTS active ---
--- RTS inactive ---
rl␀l��|␀�l�|␃␌␄␌�␌l�␌#|����␒�r�#�␌#�p␡�n�loN���␌c␜p�$r$rlp�o�␐␃␄␌�␄l␄␌␌␄␌␄b␌N�|␂l�␄��␄b��no�␃l��␀$`␃�␓2Nn␄$`␂␎␃or�ےn␄␌2�� ␂p�N�␐␂␌␌r�����␄␌␄␌c␄o�|␃���␌␄b��oN�␃␄␌d ␃�␛2No$�␜rl␏␂n{���N␌␄c��␎d␎sےn␄␌#�� ␂r$�#�no���p|␌�b␜␀lc�␛op`␛b�␒b�b��n␀��o�b�Nbp~�o␏l␜�n�␂n�o~␒#␄␌␌�␜␀$lp"�� ␃␜�n�␃o$lo���␄␌␄l��~� ␃␜�n�␂n�␛���␐b␌␄␄��␄l$l␡{l�b�on�␀�l�␛oNo>␀� B�␃��o�␃N␎lrl␌�on��|�␂�|␡�n␌�b␎$��␂�␓�N�|␛r��o|�␌d␄ll$`␃��s�p���: 0080166 Commandhandlercommand status registered
80576 Commandhandlercommand echo registered
80875 Commandhandlercommand help registered
81178 Commandhandlercommand debugon registered
85161 Commandhandlercommand debugoff registered
89338 Commandhandlercommand command registered
Welcome to the Stream Command executor
96707 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
10101328 FS check res: -10024
10101552 trying to mount spiffs at 0x00100000, length 65536
10101747 fs.start: size:64 Kb, offset:0x100000

10103860 mount res: 0

save 1

Full cmd is: save 1
Cmd1: save, cmd2: 1
26884600 api_spiffs_erase
26906674 api_spiffs_erase
26931661 Bytes written to fs: 10
save 1

Full cmd is: save 1
Cmd1: save, cmd2: 1
30227023 Bytes written to fs: 10
--- RTS active ---
--- RTS inactive ---
s$␀l��|␀�l�|␃␌␄␌�␄l�␌c|��␂�p␃�s��␟�␄"��no�loN���␌c␜p�$r$rlp�n�␐␂␌␌�␌l␄␌␄␌␌␄b␄N�|␂l�␄��␄"��no�␂l��␀l ␂�␛␒no␄l`␂␎␃or�ےn␌␄␛�`␂p�o�␐␃␄␌{�ܜ��␌␄␄␌#␌o�|␃����␄b��oo�␃␌␄l ␂�␒␛oNl�l`␂␎␃or�ےn␌␄c��␎l␎rےn␌␄c��␎drl�b�oN�␀��p|␄�|�␄lc�␛op`␛b�2b�B��o␀��n�#�ncp~�o␏l␜�N�␂n�n␡␛b␌␌␄�␄l$pb��`␃␜�n�␂ndlo��c␌␄␌l�ld␏l␜�N�␂n�␒����>␀␌␌�␌dll~rl�b�on�␀�l����on␟␀�␛o܌�n�␃o␏lr$␄�oo��|�܌�n�␀␌�#␎l��␂�p␃�N천p��o|�␌l␄$ll ␂��r�l��: 0080381 Commandhandlercommand status registered
80791 Commandhandlercommand echo registered
81090 Commandhandlercommand help registered
81392 Commandhandlercommand debugon registered
85377 Commandhandlercommand debugoff registered
89553 Commandhandlercommand command registered
Welcome to the Stream Command executor
96922 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
10101543 FS check res: -10024
10101766 trying to mount spiffs at 0x00100000, length 65536
10101962 fs.start: size:64 Kb, offset:0x100000

10104072 mount res: 0

save 1

Full cmd is: save 1
Cmd1: save, cmd2: 1
26142280 Bytes written to fs: 10
save 100

Full cmd is: save 100
Cmd1: save, cmd2: 100
32530623 Bytes written to fs: 10
32538193 Bytes written to fs: 20
32545746 Bytes written to fs: 30
32553297 Bytes written to fs: 40
32560841 Bytes written to fs: 50
32568382 Bytes written to fs: 60
32575903 Bytes written to fs: 70
32583418 Bytes written to fs: 80
32590923 Bytes written to fs: 90
32598428 Bytes written to fs: 100
32605916 Bytes written to fs: 110
32613453 Bytes written to fs: 120
32619477 api_spiffs_erase
32639630 api_spiffs_erase
32662821 Bytes written to fs: 130
32670502 Bytes written to fs: 140
32678071 Bytes written to fs: 150
32685628 Bytes written to fs: 160
32693185 Bytes written to fs: 170
32700725 Bytes written to fs: 180
32708258 Bytes written to fs: 190
32715782 Bytes written to fs: 200
32723306 Bytes written to fs: 210
32730809 Bytes written to fs: 220
32738312 Bytes written to fs: 230
32745799 Bytes written to fs: 240
32753283 Bytes written to fs: 250
32760754 Bytes written to fs: 260
32768222 Bytes written to fs: 270
32775211 api_spiffs_erase
32797328 api_spiffs_erase
32821471 Bytes written to fs: 280
32828978 Bytes written to fs: 290
32836509 Bytes written to fs: 300
32844041 Bytes written to fs: 310
32851552 Bytes written to fs: 320
32859058 Bytes written to fs: 330
32866556 Bytes written to fs: 340
32874056 Bytes written to fs: 350
32881533 Bytes written to fs: 360
32889005 Bytes written to fs: 370
32896465 Bytes written to fs: 380
32903923 Bytes written to fs: 390
32911366 Bytes written to fs: 400
32918805 Bytes written to fs: 410
32926230 Bytes written to fs: 420
32933701 Bytes written to fs: 430
32939638 api_spiffs_erase
32959837 api_spiffs_erase
32984891 Bytes written to fs: 440
32992478 Bytes written to fs: 450
32999960 Bytes written to fs: 460
33007429 Bytes written to fs: 470
33014889 Bytes written to fs: 480
33022340 Bytes written to fs: 490
33029792 Bytes written to fs: 500
33037223 Bytes written to fs: 510
33044652 Bytes written to fs: 520
33052074 Bytes written to fs: 530
33059480 Bytes written to fs: 540
33066878 Bytes written to fs: 550
33074268 Bytes written to fs: 560
33081644 Bytes written to fs: 570
33089021 Bytes written to fs: 580
33095965 api_spiffs_erase
33121164 api_spiffs_erase
33150843 Bytes written to fs: 590
33158213 Bytes written to fs: 600
33165800 Bytes written to fs: 610
33173382 Bytes written to fs: 620
33180954 Bytes written to fs: 630
33188526 Bytes written to fs: 640
33196079 Bytes written to fs: 650
33203628 Bytes written to fs: 660
33211169 Bytes written to fs: 670
33218702 Bytes written to fs: 680
33226221 Bytes written to fs: 690
33233737 Bytes written to fs: 700
33241235 Bytes written to fs: 710
33248736 Bytes written to fs: 720
33256221 Bytes written to fs: 730
33263748 Bytes written to fs: 740
33269893 api_spiffs_erase
33294605 api_spiffs_erase
33322252 Bytes written to fs: 750
33329997 Bytes written to fs: 760
33337575 Bytes written to fs: 770
33345146 Bytes written to fs: 780
33352708 Bytes written to fs: 790
33360261 Bytes written to fs: 800
33367802 Bytes written to fs: 810
33375335 Bytes written to fs: 820
33382864 Bytes written to fs: 830
33390379 Bytes written to fs: 840
33397884 Bytes written to fs: 850
33405382 Bytes written to fs: 860
33412880 Bytes written to fs: 870
33420358 Bytes written to fs: 880
33427830 Bytes written to fs: 890
33434934 api_spiffs_erase
33459593 api_spiffs_erase
33486327 Bytes written to fs: 900
33493897 Bytes written to fs: 910
33501487 Bytes written to fs: 920
33509068 Bytes written to fs: 930
33516636 Bytes written to fs: 940
33524202 Bytes written to fs: 950
33531747 Bytes written to fs: 960
33539296 Bytes written to fs: 970
33546829 Bytes written to fs: 980
33554356 Bytes written to fs: 990
33561877 Bytes written to fs: 1000
--- RTS active ---
--- RTS inactive ---
rd␀lܟ|␀�$�|␂␄␌␄�␌d�␌#|��␂�2�{��␟�␄b��no�loN���␌c␜p��$r$rlp�n�␐␂␌␄�␌$␌␄␄␌␄␌c␄n�|␂l�␌��␄b��no�␃l��␀$`␃�␒␛on␌$`␃␏␂n{���o␌␌␒� ␃p�n�␐␂␌␌r�����␄␄␌␄b␌N��o���␄␄b��oN�␃␌␄l ␂�2␒ool�$`␃␏␂n{���o␌␌b�� ␃␏r��o␌␌b�� ␃{$�#�no�����b�"␜␀$b�␒np ␒c�␒#��|��n␀���2␒no"p␡�N␎d␜�o�␃o���B␄␌␄�␄ldpb��`␃␜�n�␂o$lo���␄␄␌$�d$␎l␜�n�␃o�␛���␐B␄␌␄��␌$ld~rl�b�oN�␀�l����oN␟␀�␓o���N�␂N␎drl␌�Nn��|�␂�|␡�n␌�b␎$��␃�␒�n�s��N|�␌d␄lld ␃�œn�l��: 0080305 Commandhandlercommand status registered
80716 Commandhandlercommand echo registered
81015 Commandhandlercommand help registered
81317 Commandhandlercommand debugon registered
85301 Commandhandlercommand debugoff registered
89477 Commandhandlercommand command registered
Welcome to the Stream Command executor
96846 hwsDelegateDemo instantiated, waiting for data
mode : softAP(be:dd:c2:91:cf:b1)
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
10101468 FS check res: -10024
10101691 trying to mount spiffs at 0x00100000, length 65536
10101886 fs.start: size:64 Kb, offset:0x100000

10103999 mount res: 0
mikee47 commented 5 years ago

This is probably an unrealistic test, for the flash to get in that state would probably indicate other problems so perhaps not realistic. For more robust operation, try setting SPIFFS_USE_MAGIC=1 in spiffs_config.h - further details here.

kmihaylov commented 5 years ago

@mikee47 respect for your guidance. It's very helpful.

I put #define SPIFFS_USE_MAGIC (1) and #define SPIFFS_USE_MAGIC_LENGTH 1

And now after make flashinit it works almost as expected:

  1. FS not found;
  2. Format initiated
  3. Unsuccessful mount.

I did a dump of the formatted partition and it is 0xFF everywhere, obviously there are no magic numbers.

So far, so good.

I thought to put some data (in order for the FS to put some magic numbers inside), but since the FS couldn't be mount this option is ignored.

I read spiffs_format_internal in spiffs_sming.c and I don't see the function to write some magic numbers during format.

So I guess it only wipes the flash

while( sect_first <= sect_last )
  {
    if(flashmem_erase_sector( sect_first++ ))
    {
        int percent = ++cur * 100 / total;
        if (percent > last)
            debugf("%d%%", percent);
        last = percent;
    }
    else
    {
        ETS_INTR_UNLOCK();
        return false;
    }
  }

UPDATE: I just found the correct procedure for initializing SPIFFS with magics: https://github.com/pellepl/spiffs/wiki/Using-spiffs

If SPIFFS_USE_MAGIC is enabled, mounting procedure will look for magic. Recommended formatting procedure is as point two above. Recapped:

    Call SPIFFS_mount

    If SPIFFS_mount fails with SPIFFS_ERR_NOT_A_FS, keep going. Otherwise, call SPIFFS_unmount

    Call SPIFFS_format

    Call SPIFFS_mount again.

Well, I tried to modify the code as pointed by the author but with no luck (FS just coudn't be mount the second time.

Currently I plan to neglect USE_MAGIC and to replace this code:

  if (dat == UINT32_MAX)
  {
      debugf("First init file system");
      spiffs_format_internal(cfg);
      writeFirst = true;
  }

  int res = SPIFFS_mount(&_filesystemStorageHandle,
    cfg,
    spiffs_work_buf,
    spiffs_fds,
    sizeof(spiffs_fds),
    spiffs_cache,
    sizeof(spiffs_cache),
    NULL);
  debugf("mount res: %d\n", res);

  if (writeFirst)
  {
      file_t fd = SPIFFS_open(&_filesystemStorageHandle, "initialize_fs_header.dat", SPIFFS_CREAT | SPIFFS_TRUNC | SPIFFS_RDWR, 0);
      SPIFFS_write(&_filesystemStorageHandle, fd, (u8_t *)"1", 1);
      SPIFFS_fremove(&_filesystemStorageHandle, fd);
      SPIFFS_close(&_filesystemStorageHandle, fd);
  }

with this one:

file_t fd = SPIFFS_open(&_filesystemStorageHandle,"initialize_fs_header.dat",SPIFFS_RDONLY,0);
  SPIFFS_close(&_filesystemStorageHandle, fd);
    if(fd < 0) {
        debugf("No initialized SPIFFS volume found. Initializing FS now...");
        file_t fd = SPIFFS_open(&_filesystemStorageHandle, "initialize_fs_header.dat", SPIFFS_CREAT | SPIFFS_TRUNC | SPIFFS_RDWR, 0);
        SPIFFS_write(&_filesystemStorageHandle, fd, (u8_t *)"1", 1);
        //SPIFFS_fremove(&_filesystemStorageHandle, fd);
        SPIFFS_close(&_filesystemStorageHandle, fd);
    } else {
        debugf("It seems that the mounted SPIFFS is properly inited.");
    }

However as indicated in my previous post, if there is some previous data on the flash (wiped flash is all 0xFF as I undestand?) it may lead to unusuable filesystem.

So it is pretty good to erase the flash before use.

slaff commented 4 years ago

@kmihaylov Can you fetch the latest develop version and check if you still have problems with spiffs?