Reading and writing files

[YonatanHanan]

Hello, Today I came across a very strange problem, I am using ESP-12.

I have the following code http://pastebin.com/vzUA8L2G is actually reads the file counter.lua (it saves a numeric value as a string) increases the value stored in it and saves it back to the counter.lua file.

After about 2000 requests to the file (reading and writing) it begins to go crazy. It clears the value and then when I run the code again it resets the ESP.

the serial :

Not a valid number
0
7!a!ˆCN�AָB”�ָV0]�A�צנ
NodeMCU 0.9.5 build 20150318  powered by Lua 5.1.4
lua: cannot open init.lua
>

It does this in a loop. what do you think is the problem?

[marcoskirsch]

In my experience the Fike system is very unstable and writing to it many times ends up corrupting it and needing format. I haven't used recent builds of the firmware though.

My advice is not to write files more than the absolute minimum necessary and rely on servers for your logging needs.

Others may have more up to date information though.

[TerryE]

Have a look at the SPIFFS documentation and the WP page on SPI Flash. What you are doing is a pathological use case. Not advised to say the least. So +1 on Marcos' advice. Don't rely on SPI Flash as high frequency update non-volatile storage. Use RA and backup to the network or add an RTC and backup to that.

This is an architectural characteristic and not really a bug of the firmware, IMO.

[YonatanHanan]

Is there a correct and safe way to use the file system as a method to keep data in a file?

[TerryE]

NandFlash and EEPROM sit somewhere between WORM storage and conventional RAM in that it can be read almost indefinitely but the normal write process involves erasing and rewriting the containing sector, and these erase cycles are limited. It is great for storing information that only chances occasionally such as program or configuration storage. Ii has very limited life if uses as pure non-volatile RAM. If you need to store non-volatile state then write it off over the network or add battery backed RAM such as on an RTC.

Another compromise is to keep a counter in RAM but echo it off-chip over the network and then request it back over the network to the initialise the count correctly on reboot.

[eku]

Does the dev120 branch provide access to the RTC storage?

[TerryE]

An RTC is an add-on component. I was thinking of something like of something like a DS1307 I2C Real Time Clock which has 56 bytes of NV RAM. You can pick these modules up for a few $. There's no GPIO pin overhead if you are already running I2C. And there's already I2C support so you can talk to it with a Lua module, though you might want to add your own C wrapper library.

But on a wider note, this issue really falls the "operating characteristics of SPI Flash" and as far as the firmware goes, this is in the "can't or won't fix" category. So unless anyone comes up with a valid reason not to, I will close this issue tomorrow.

[Aeprox]

I think @eku is refering to the RTC memory that's present onboard? Implemented by #545.

The RTC in the ESP8266 contains memory registers which survive a deep sleep, making them highly useful for keeping state across sleep cycles. Some of this memory is reserved for system use, but 128 slots (each 32bit wide) are available for application use. This module provides read and write access to these.

This only works across deep-sleep cycles though (I presume), so you'd still need an external battery powered non-volatile memory for it to work after power is removed from the module.

Though I fully agree this is not a bug or issue related to the firmware.

[TerryE]

Durrhhh. Caught in my own little world. Missed this one entirely and very valuable. Thanks again to Johnny and to @Aeprox for point this out.. This RAM isn't battery backed, but it does survive deep sleep and as long as there's power to the esp8266 then this should work fine -- but you've only got 56 bytes to play with which should be enough for most embedded applications.

[eyaleb]

RE "you've only got 56 bytes": not accurate. AFAIR there are 64 words (4 bytes each) for the SDK followed by 128 words for the user. Not bad at all for preserving state but not enough for general storage. For logging real data locally I would consider attaching an SD socket.

In 0.9.5 the RTC mem implementation in the SDK is buggy, I am really waiting for the SDKv1 based fw.

[va7ta]

I do not disagree with the recommendations here but I noticed the eeprom on my board is a GigaDevice 25Q41Bx which is rated for minimum 100,000 read/write cycles. One would think this memory should easily handle 2000 cycles without failing. I wonder if possibly there is a bug remaining that causes multiple write cycles to take place each time data is saved?