Add support for exporting total (energy used) and total_returned (energy returned) in Shelly 3EM / Support Shelly Pro 3 EM

[MatiasLGonzalez]

Hello 👋 first of all, awesome exporter! Though I've noticed that it doesn't support exporting the total energy and total returned energy metrics, I believe this would be fairly easy to implement since it's already part of the /status JSON object. If I get some free time I will try to tackle this issue.

[lukassoo]

Hi 🙂 Thanks for using the exporter.

I have mentioned this in the readme:

I left out the total power since it resets every time the device restarts so there is not much use of it. You are far better off using the current power metric and just calculating the total power used. If someone actually has a good use case for the total power - let me know, I will add it.

Could you tell me why you would prefer using the totals that reset?

[MatiasLGonzalez]

Oh, sorry for my misunderstanding, I thought that only applied to the shelly plug, funny to know that the Energy Meter doesn't actually keep track of the energy consumption after a reboot 😃

Maybe the exporter could wright the last value to a file with a timestamp, and if it detects that the value given by the shelly is below what it has written to that file, it adds that as a delta, and also use the timestamp and the shelly's uptime to keep track of who has the most recent and correct value, another approach could be to use the emeter csv file of energy consumption, but that will take more time to get the file each time as the file grows. I know this is kind of sketchy and beats the porpouse of a exporter, whose task should only be to receive data and export it as prometheus metrics.

[lukassoo]

I am sorry, it would seem that you are right. I remember testing the totals and seeing them reset on restart with my Shelly Plug S but I didn't test it with the 3EM and just assumed it would work the same way. So when you said that it is supposed to be an actual Energy Meter I took a good look at the documentation and would you look at that, I was wrong:

shellies/shellyem3-/emeter/\<i>/total total energy in Wh (accumulated in device's non-volatile memory) shellies/shellyem3-/emeter/\<i>/total_returned total energy returned to the grid in Wh (accumulated in device's non-volatile memory)

I will be updating the exporter to add these two exported values. It may take a few days but it will be a big update to all exporters with fixes and improvements from #4 I am also thinking of writing a Wiki instead of infinitely adding to the already long readme 😄

[MatiasLGonzalez]

I actually thought you were correct since I did a quick google of "shelly energy reset after reboot" and they all confirmed what you said but I didn't realize that they were all "PM" devices and plugs, I'm glad you took the time to investigate further. I've gone ahead and actually rebooted my 3EM and lo and behold, it does, in fact, keep track of the energy consumption after a reboot :) 100% real no fake. Thanks again for all you effort!

[lukassoo]

I have exposed the missing metrics and for the time being, released all changes under containers with development tags. On DockerHub you can download and use:

• development
• development-armv7
• development-armv8

Depending on your platform to test if everything works as intended.

I didn't have the time to set up proper build automation so I built and pushed everything by hand so if something doesn't work let me know. After my home server had some major issues it is now back online and is also testing the new image.

The configs only get generated if no existing config is found. This means that to get the new configuration filed in there you either have to write them in yourself or remove/rename the config so the container generates a new one and copy over values from the old one. If you don't intend to ignore any metrics you can just leave it as is 🙂

[Sylweter]

I have just launched development version of 3em exporter, but it is still connecting to /status, instead /rpc endpoint. RPC endpoint is working: curl -X POST -d '{"id":0,"method":"Shelly.GetStatus"}' http://${SHELLY}/rpc {"id":1,"src":"shellypro3em-idcleared","result":{"ble":{},"cloud":{"connected":true},"em:0":{"id":0,"a_current":0.727,"a_voltage":239.2,"a_act_power":-163.1,"a_aprt_power":173.9,"a_pf":0.93,"a_freq":50.0,"b_current":1.500,"b_voltage":239.7,"b_act_power":288.5,"b_aprt_power":359.5,"b_pf":0.81,"b_freq":50.0,"c_current":0.566,"c_voltage":239.8,"c_act_power":-99.4,"c_aprt_power":135.6,"c_pf":0.79,"c_freq":50.0,"n_current":null,"total_current":2.793,"total_act_power":26.092,"total_aprt_power":669.001, "user_calibrated_phase":[]},"emdata:0":{"id":0,"a_total_act_energy":24589.01,"a_total_act_ret_energy":17556.35,"b_total_act_energy":294736.47,"b_total_act_ret_energy":4114.58,"c_total_act_energy":111186.55,"c_total_act_ret_energy":7460.64,"total_act":430512.03, "total_act_ret":29131.57},"eth":{"ip":null},"modbus":{},"mqtt":{"connected":false},"sys":{"mac":"abc_cleared","restart_required":false,"time":"11:02","unixtime":1705831321,"uptime":591,"ram_size":241080,"ram_free":99176,"fs_size":524288,"fs_free":172032,"cfg_rev":21,"kvs_rev":0,"schedule_rev":0,"webhook_rev":0,"available_updates":{},"reset_reason":3},"temperature:0":{"id": 0,"tC":26.2, "tF":79.2},"wifi":{"sta_ip":"a.b.c.d_cleared","status":"got ip","ssid":"ssid_cleared","rssi":-82},"ws":{"connected":false}}} Should I use some kind of configuration option in config file?

[lukassoo]

I see, you are actually using the Shelly Pro 3 EM model. That is another Gen 2 device that would require a new importer that would use the RPC endpoint. Same situation as with #4 where I added support for the Shelly Plus Plug S. Since the web socket handler is now written I will only have to write parsing for the right component. From what I read on the docs it should be the EMData component that should be requested. Could you make the device available for me to test a new exporter ? We can share private data such as IPs through a private channel.

[Sylweter]

No problem, I have sent an invite in Discord to lukassoo user. Please send me your IP adres.

[MatiasLGonzalez]

It seems to be working fine! :) However on first boot it seems that it won't exit after writing the config file:

[lukassoo]

@MatiasLGonzalez Did you perhaps add "--restart always" to the run command? That could cause a restart loop attempting to start it right after it stops. However, it should not happen anyways since Docker doesn't restart a container if it doesn't successfully run for 10 seconds and just writing a config should be much faster.

[MatiasLGonzalez]

@MatiasLGonzalez Did you perhaps add "--restart always" to the run command? That could cause a restart loop attempting to start it right after it stops. However, it should not happen anyways since Dicker doesn't restart a container if it doesn't successfully run for 10 seconds and just writing a config should be much faster.

Yes, indeed I forgot I set my restart policy to unless-stopped, all good then!

[lukassoo]

No issues have appeared in quite some time so I am closing this one. Docker images have been updated and I will be uploading a Grafana dashboard and updating the readme/wiki hopefully soon.