Total consumption lost when rebooting or updating

[mariusbach]

Is there a way to preserve the total consumption value when rebooting or updating an energy metering plug/socket?

Again, implementing in firmware would save every user from programming his own routine for this.

[mariusbach]

If there is a better place to ask questions like these, please let me know!

[openshwprojects]

@mariusbach I can add saving of total consumption if you want. I can split it into "This session" and into "Ever".

@valeklubomir , what do you think? I could use the same place where LED state is saved or where channels are saved.

I am not at home now, but I can look into it when I come back. Great suggestion, @mariusbach !

[mariusbach]

@openshwprojects That'd be awesome!

While you're at it, could you also report on a separate MQTT topic a daily value which resets at midnight? I had a similar question at #316. While thinking about it, because the total Wh are counted, why not create a second counter which resets at midnight...

[valeklubomir]

As I mentioned in #316 I have some doubts using flash for storing data. I don't know if BK7231 or other platforms have dedicated dataflash, capable of 100,000 write cycles or more. But regular program flash endurance is usually guaranted for max 10,000 write cycles. First of all use serarate block, not shared with configuration. Second I would use save logic build on minimal time and minimal threshold for change. Reducing write cycles. I am working on function model which stores and resets consumption at mignight and send it to MQTT cloud. I have not yet figured out why after some time I loose Wifi connection and even with ping watchdog, it does not reconnect to Wifi.

[mariusbach]

I have not yet figured out why after some time I loose Wifi connection and even with ping watchdog, it does not reconnect to Wifi.

Indeed, same here with one of 3 plugs. The others seem to work fine for now. The one disconnecting has nothing special, no special characters anywhere or different power consumption or similar... also it's close to the access point, so it cannot be loss of wifi signal.

[openshwprojects]

@valeklubomir two things.

1. the 'flash vars' system, the one created by @btsimonh , was designed to optimize flash usage (it is used to save reboot count and channel states). It uses separate block, different than configuration (that cfg doesn't change often, but flashvars do change often). Futhermore, it uses sector by sector gradually instead of writing to a single place all the time (as far as I know, @btsimonh designed it)
2. in SPI direct flash mode, BK identifies itself as EN25QH16B - data sheet here: https://datasheetspdf which has  Minimum 100K endurance cycle

[openshwprojects]

@valeklubomir take a look here: https://github.com/openshwprojects/OpenBK7231T_App/commit/8b7ba205441dc3e8bb464c2a81cbebb7a96d0b4c it's a sample, the slots were in theory dedicated for channels, but there are hardly any devices with 12 channels, I barely managed to find a 4 relays (4 channels) controller My commit saves a short (2 bytes), but you can split 4 bytes into two shorts, as you certainly know. Tell me if you want to take from here or if you want me to add that autosave of power consumption (total), with some kind of "Clear" button on GUI and of course only saving from time to time, idk, save every 2 minutes if change larger than X... etc etc

[valeklubomir]

Then I would need to save following parameters:

float TotalConsuption; float DayConsumption; float YersterdayConsumption[1]; int actualDay; long SaveCounter;

If neccassary few more float variables. Consumption of last hour will be lost. Threshold for consumption save ~1Wh Midnight save.

[openshwprojects]

@valeklubomir this is the current structure layout: The larger the structure, the faster is flash wear. I usually would just try to use "MAX_RETAIN_CHANNELS" array, but keep at least first 4 channels for real channels (there are some 4 channel devices with power metering, like a power strip with sockets etc). It seems that your required data would not fit into the existing structure, even if we reuse almost all of the fields. We would have to consider how to get around it. Do we just increase the structure, let's say, from the current 32 bytes to the 64 bytes and double the flash wear speed? Or maybe we alter system to allow flash vars to be variable-sized? I will think about it tomorrow.

EDIT: if we double structure size, from 32 bytes to 64 bytes, you get 32 bytes for you and that's 8 floats or integers (assuming 4 bytes), or 6 floats/integers and one long (long is 8 bytes, double size of integer).

[valeklubomir]

@openshwprojects changing size from 32 to 64 shold be pretty simple. As I understand the logic it should adapt automatically, because vars are stored in sequence and when reaching end of storage area, whole area is erased and starts from beginning. Only 2 from 5 platforms have real storage present others return 0. Will you do it or should I try?

[valeklubomir]

I am doing it.

[mariusbach]

@valeklubomir @openshwprojects I'm so very happy you're looking into this, really. Thanks a lot!

[openshwprojects]

@valeklubomir good, do it, just make sure that the struct you add is of size 32 bytes so we get a nice line up of whole current structure to 64 bytes. Make sure with sizeof( ) . Maybe make that struct public and easily accessible.

[silbor]

Should we store "startdate" or "bootdate" as well? If not, you do not know what total energy consumption refers to.

[mariusbach]

Good idea, for my use case I wouldn't need it, though. But while you're at it, I'd say yes, if the struct is big enough to store that as well.

[valeklubomir]

Place is there. I could add field with date/time when statistics was last time cleared.

[mariusbach]

Guys, thanks so much, I have upgraded to include the new functions. Total consumption is saved, perfect.

However, I'm not sure how to use the daily statistics reporting. Will the MQTT message be sent only at midnight? Can I set a timezone or offset, and where?

Also since I have the plugs, it tells me that periodic statistics are disabled and I guess I have to enable them. Adding the line to the startup command doesn't work.

Is this the right code? Startup command: backlog startDriver BL0942; SetupEnergyStats;

Or do you expect this: backlog startDriver BL0942, SetupEnergyStats or backlog startDriver BL0942 && SetupEnergyStats

I'm quite lost when it comes to put more than 1 command there, what the syntax is. Or even how to write an autoexec-file if wanting to use that way of doing it. I feel the wiki pages would need an update for this case?

[valeklubomir]

For daily statistics I had to execute multiple commands:

• Enable driver
• Enable Energy statistics
• Enable NTP for time synchrozation
• Setup time zone offset.

"backlog startDriver BL0942; startDriver NTP; ntp_timeZoneOfs 2"

New additional MQTT topic will be activated:

• */energycounter_yesterday/get
• */energycounter_today/get

If you want lasthour topic "*/energycounter_last_hour/get" add: "SetupEnergyStats 1 30 120 1;"

We have updated README section for help with commands.

[mariusbach]

@valeklubomir and @openshwprojects this worked, thanks so much! I've noticed it's a lot of code because you thought of many things, so thanks again for the hard work.

I suggest we close this one, shall I?

Now just one issue remains for me personally, the hard reboot requirement on some plugs after some time :) #203

[openshwprojects]

Yes, I think its sovled. Again, big thanks for @valeklubomir for his continuous helpful contribution!