energy/meter - Githubissues

gisbertg commented 1 year ago

Could you take the entities for an Energy/Meter from Grott? I still have a SDM630 installed in my installation. It is read out by Grott but unfortunately not displayed by you.

MQTT jsonmsg: {"device": "GPG0A271VP", "time": "2023-05-16T06:50:30", "buffered": "no", "values": {"datalogserial": "GPG0A271VP", "pvserial": "QCL0CGB0BC", "voltage_l1": 2323, "voltage_l2": 2332, "voltage_l3": 2330, "current_l1": 4, "current_l2": 8, "current_l3": 8, "act_power_l1": -334, "act_power_l2": -1208, "act_power_l3": -1122, "app_power_l1": 1125, "app_power_l2": 2029, "app_power_l3": 1924, "react_power_l1": 0, "react_power_l2": -1215, "react_power_l3": -1246, "powerfactor_l1": -305, "powerfactor_l2": -610, "powerfactor_l3": -582, "pos_rev_act_power": -2664, "pos_act_power", "rev_act_power": -2664, "app_power": 5049, "react_power": -2573, "powerfactor": -529, "frequency": 499, "L1-2_voltage": 4032, "L2-3_voltage": 4038, "L3-1_voltage": 4030, "pos_act_energy": 2931, "rev_act_energy": 5289}}
- Grott MQTT topic used : energy/meter
- MQTT message message sent
- Grott Send data to PVOutput disabled
- Grott Send data to Influx disabled
- Grott extension processing disabled

Translated with www.DeepL.com/Translator (free version)

muppet3000 commented 1 year ago

Hi @gisbertg

I think you're referring to this set of values from grott: https://github.com/johanmeijer/grott/blob/master/grottconf.py#L1052-L1092

I need to work out what the 'human readable' versions of all these names are, once I've done that I'll be able to map them through for you.

muppet3000 commented 1 year ago

I've asked the developer of Grott for this information here: https://github.com/johanmeijer/grott/discussions/340

If he doesn't come back to me I'll have to try and do it a different way, perhaps if you'd be willing to share access to your growatt account I could look it up that way. But let's see what the developer comes back with first.

@shocker2 - I've noticed you have one of these meters too based on your post here: https://github.com/johanmeijer/grott/issues/339 perhaps you'd be willing to assist with the correct names for these fields?

gisbertg commented 1 year ago

That's exactly what they are. You can also learn more about it here. https://github.com/johanmeijer/grott/issues/88 As shown here would be perfect I think. https://github.com/johanmeijer/grott/issues/88#issuecomment-1013877389

Of course I don't know how many users have this setup, but there are quite a few I think.

muppet3000 commented 1 year ago

@gisbertg thanks that's a great start.

Can I confirm that you do get some of the information showing e.g. pvserial? You're just missing a load of the useful information e.g. all the data values.

gisbertg commented 1 year ago

32 entities are shown to me in Homeassistant.

muppet3000 commented 1 year ago

@gisbertg what does it show on the device page please? Settings -> Devices & Services -> Grott

How many devices do you have?

What information is shown when you click on each device?

shocker2 commented 1 year ago

@muppet3000 happy to help, not sure what the ask is :) can you please provide additional info? I do have SDM630 Modbus v3 as mettering and in HA Grott I do have two devices. One is the invertor that we have discussed with all the values, the other is the data logger with two values, Datalog Serial and PV Serial. Not sure if there is in Grott logs anything related to the meter as I cannot see anything related to the export power.

gisbertg commented 1 year ago

egguy commented 1 year ago

Grott manage the SDM630. But there's can only be one per install

The SDM630 is logged with its message. The name of the devices is SDM630. @muppet3000 if you want to find the keys in the message, the values exported can be found in grottconf.py:1094

PS: sorry for the noise. You seems to have this info.

shocker2 commented 1 year ago

I have attached the log file if that helps

grott.log

muppet3000 commented 1 year ago

I think everyone may have confused what I was asking for, I know what all the keys are for this data type, however in the Integration I map them on to nice human readable values e.g.

pvserial becomes PV Serial

However, I don't know what those human readable names should be for the SDM630 type for example what is the human friendly version of act_power_l1 - I don't know what that means, or what unit it is in.

Separately, it's strange that @gisbertg is showing just 1 device when @shocker2 is showing 2. @shocker2 - what are the two devices that you have registered in Home Assistant? Is one of them the SDM630 with just a very small amount of information e.g. pvserial and dataloggerserial?

shocker2 commented 1 year ago

@shocker2 - what are the two devices that you have registered in Home Assistant? Is one of them the SDM630 with just a very small amount of information e.g. pvserial and dataloggerserial?

No, one is the invertor (with all the values) and the other one is the data logger. I have identified them based on the serial number.

muppet3000 commented 1 year ago

@shocker2 - what are the two devices that you have registered in Home Assistant? Is one of them the SDM630 with just a very small amount of information e.g. pvserial and dataloggerserial?

No, one is the invertor (with all the values) and the other one is the data logger. I have identified them based on the serial number.

Would it be possible to get a screen grab of what is shown on each of the 'device' pages for them?

I'm sure I've seen the SDM mentioned in your log files so I'm surprised it's not showing up in HA at all.

gisbertg commented 1 year ago

By the way, why do I only have 1 device in HA? I have a Mod9000TL3-XH and a Shine Lan-x Stick. But in HA only the inverter is shown.

muppet3000 commented 1 year ago

By the way, why do I only have 1 device?

That was my next question, in the grott logs are all of your devices showing as being output to MQTT?

When you added the Grott Integration to Home Assistant did you leave it as the default '+' setting so that it includes all devices?

shocker2 commented 1 year ago

Would it be possible to get a screen grab of what is shown on each of the 'device' pages for them?

I'm sure I've seen the SDM mentioned in your log files so I'm surprised it's not showing up in HA at all.

muppet3000 commented 1 year ago

Thanks @shocker2 - That second device (ending 3VK) is the SDM630 device, when I know what all the values should appear as for human values I'll be able to add them in and that will all be populated as well. I just need to get the information on what the human readable values are. I'll try and take a look after work this evening.

shocker2 commented 1 year ago

Thanks @shocker2 - That second device (ending 3VK) is the SDM630 device, when I know what all the values should appear as for human values I'll be able to add them in and that will all be populated as well. I just need to get the information on what the human readable values are.

I'll try and take a look after work this evening.

Thank you! Busy Wednesday as well 😀 let me know if you need help when adding the values to identify what they are based on what I have on Shineapp.

muppet3000 commented 1 year ago

Thanks @shocker2 - That second device (ending 3VK) is the SDM630 device, when I know what all the values should appear as for human values I'll be able to add them in and that will all be populated as well. I just need to get the information on what the human readable values are. I'll try and take a look after work this evening.

Thank you! Busy Wednesday as well 😀 let me know if you need help when adding the values to identify what they are based on what I have on Shineapp.

That would be REALLY valuable, if you could send me a screengrab of what those values appear as on the shineapp then I should be able to work it out. I'm 99% sure that I've written the integration in a way that means I can change the names without losing the history of the sensor so if I don't get it perfect first time I can tweak it.

gisbertg commented 1 year ago

log.txt

To me it looks like he is transferring everything to Mqtt.

muppet3000 commented 1 year ago

Thanks @gisbertg - Yes it looks like they're all being published to MQTT. The HA integration should pick them up automatically if you've used the + in the configuration of the integration:

gisbertg commented 1 year ago

Thats mine ...

muppet3000 commented 1 year ago

ok in which case it should be picking everything up, they all get published to the same place in MQTT and the integration should pick them all up automatically.

I'll have to have a think about it. You could try removing and then re-adding the integration, but I don't think that would fix it.

gisbertg commented 1 year ago

Have reinstalled it and now 2 entities less. :-(

gisbertg commented 1 year ago

The funny thing is that he seems to know both devices, at least he knows both serials.

shocker2 commented 1 year ago

That would be REALLY valuable, if you could send me a screengrab of what those values appear as on the shineapp then I should be able to work it out. I'm 99% sure that I've written the integration in a way that means I can change the names without losing the history of the sensor so if I don't get it perfect first time I can tweak it.

Values in Grott:

grott        |   - date-time:  2023-05-17T08:42:48
grott        |   - Grott values retrieved:
grott        |           -  datalogserial        :  the_3VK
grott        |           -  pvserial             :  the_07Y
grott        |           -  voltage_l1           :  217.5
grott        |           -  voltage_l2           :  238.5
grott        |           -  voltage_l3           :  235.7
grott        |           -  Current_l1           :  20.6
grott        |           -  Current_l2           :  3.9
grott        |           -  Current_l3           :  1.2
grott        |           -  act_power_l1         :  4496.3
grott        |           -  act_power_l2         :  149.0
grott        |           -  act_power_l3         :  95.8
grott        |           -  app_power_l1         :  4512.2
grott        |           -  app_power_l2         :  949.0
grott        |           -  app_power_l3         :  305.9
grott        |           -  react_power_l1       :  -138.3
grott        |           -  react_power_l2       :  -935.3
grott        |           -  react_power_l3       :  -286.0
grott        |           -  powerfactor_l1       :  1.0
grott        |           -  powerfactor_l2       :  0.2
grott        |           -  powerfactor_l3       :  0.4
grott        |           -  pos_rev_act_power    :  4741.1
grott        |           -  pos_act_power        :  4741.1
grott        |           -  rev_act_power        :  4741.1
grott        |           -  app_power            :  4965.3
grott        |           -  react_power          :  -1346.3
grott        |           -  powerfactor          :  1.0
grott        |           -  frequency            :  49.8
grott        |           -  L1-2_voltage         :  395.1
grott        |           -  L2-3_voltage         :  410.8
grott        |           -  L3-1_voltage         :  392.7
grott        |           -  pos_act_energy       :  22737.1
grott        |           -  rev_act_energy       :  179.7
grott        |           -  pos_act_energy_kvar  :  16.5
grott        |           -  rev_act_energy_kvar  :  6673.4
grott        |           -  app_energy_kvar      :  23873.3
grott        |           -  act_energy_kwh       :  22916.8
grott        |           -  react_energy_kvar    :  6689.9
grott        |   - MQTT jsonmsg: 
grott        |                   {"device": "the_3VK", "time": "2023-05-17T08:42:48", "buffered": "no",
grott        |                   "values": {"datalogserial": "the_3VK", "pvserial": "the_07Y",
grott        |                   "voltage_l1": 2175, "voltage_l2": 2385, "voltage_l3": 2357, "Current_l1":
grott        |                   206, "Current_l2": 39, "Current_l3": 12, "act_power_l1": 44963,
grott        |                   "act_power_l2": 1490, "act_power_l3": 958, "app_power_l1": 45122,
grott        |                   "app_power_l2": 9490, "app_power_l3": 3059, "react_power_l1": -1383,
grott        |                   "react_power_l2": -9353, "react_power_l3": -2860, "powerfactor_l1": 999,
grott        |                   "powerfactor_l2": 183, "powerfactor_l3": 354, "pos_rev_act_power": 47411,
grott        |                   "pos_act_power": 47411, "rev_act_power": 47411, "app_power": 49653,
grott        |                   "react_power": -13463, "powerfactor": 962, "frequency": 498, "L1-2_voltage":
grott        |                   3951, "L2-3_voltage": 4108, "L3-1_voltage": 3927, "pos_act_energy": 227371,
grott        |                   "rev_act_energy": 1797, "pos_act_energy_kvar": 165, "rev_act_energy_kvar":
grott        |                   66734, "app_energy_kvar": 238733, "act_energy_kwh": 229168,
grott        |                   "react_energy_kvar": 66899}}
grott        |   - Grott MQTT topic used : energy/growatt
grott        |   - MQTT message message sent
grott        |   - Grott Send data to PVOutput disabled 
grott        |   - Grott Send data to Influx disabled 
grott        |   - Grott extension processing disabled 
grott        |

pos_act_energy -> imported from grid total since installation rev_act_energy -> exported to the grid since the installation act_power_l1 + act_power_l1 + act_power_l1 -> Import from the grid (4.74kW on picture, I guess this is the last read not a counter). app_power no idea what's the difference between the act power. rev_act_energy -> exported to grid pos_rev_act_power, pos_act_power, rev_act_power -> import from the grid (not sure what's the difference between them)

There are other values that I'm unable to identify them :)

Latest snip:

shocker2 commented 1 year ago

grott        |   - date-time:  2023-05-17T09:24:21
grott        |   - Grott values retrieved:
grott        |           -  datalogserial        :  the_3VK
grott        |           -  pvserial             :  the_07Y
grott        |           -  voltage_l1           :  218.7
grott        |           -  voltage_l2           :  228.6
grott        |           -  voltage_l3           :  227.3
grott        |           -  Current_l1           :  24.0
grott        |           -  Current_l2           :  14.3
grott        |           -  Current_l3           :  12.2
grott        |           -  act_power_l1         :  5270.3
grott        |           -  act_power_l2         :  3187.9
grott        |           -  act_power_l3         :  2750.0
grott        |           -  app_power_l1         :  5235.8
grott        |           -  app_power_l2         :  3256.3
grott        |           -  app_power_l3         :  2742.5
grott        |           -  react_power_l1       :  -129.1
grott        |           -  react_power_l2       :  -776.1
grott        |           -  react_power_l3       :  -378.3
grott        |           -  powerfactor_l1       :  1.0
grott        |           -  powerfactor_l2       :  1.0
grott        |           -  powerfactor_l3       :  1.0
grott        |           -  pos_rev_act_power    :  11208.2
grott        |           -  pos_act_power        :  11208.2
grott        |           -  rev_act_power        :  11208.2
grott        |           -  app_power            :  11241.4
grott        |           -  react_power          :  -1331.3
grott        |           -  powerfactor          :  1.0
grott        |           -  frequency            :  49.9
grott        |           -  L1-2_voltage         :  387.6
grott        |           -  L2-3_voltage         :  394.8
grott        |           -  L3-1_voltage         :  386.3
grott        |           -  pos_act_energy       :  22742.6
grott        |           -  rev_act_energy       :  179.7
grott        |           -  pos_act_energy_kvar  :  16.5
grott        |           -  rev_act_energy_kvar  :  6674.4
grott        |           -  app_energy_kvar      :  23878.9
grott        |           -  act_energy_kwh       :  22922.3
grott        |           -  react_energy_kvar    :  6691.0
grott        |   - MQTT jsonmsg: 
grott        |                   {"device": "the_3VK", "time": "2023-05-17T09:24:21", "buffered": "no",
grott        |                   "values": {"datalogserial": "the_3VK", "pvserial": "the_07Y",
grott        |                   "voltage_l1": 2187, "voltage_l2": 2286, "voltage_l3": 2273, "Current_l1":
grott        |                   240, "Current_l2": 143, "Current_l3": 122, "act_power_l1": 52703,
grott        |                   "act_power_l2": 31879, "act_power_l3": 27500, "app_power_l1": 52358,
grott        |                   "app_power_l2": 32563, "app_power_l3": 27425, "react_power_l1": -1291,
grott        |                   "react_power_l2": -7761, "react_power_l3": -3783, "powerfactor_l1": 999,
grott        |                   "powerfactor_l2": 971, "powerfactor_l3": 990, "pos_rev_act_power": 112082,
grott        |                   "pos_act_power": 112082, "rev_act_power": 112082, "app_power": 112414,
grott        |                   "react_power": -13313, "powerfactor": 992, "frequency": 499, "L1-2_voltage":
grott        |                   3876, "L2-3_voltage": 3948, "L3-1_voltage": 3863, "pos_act_energy": 227426,
grott        |                   "rev_act_energy": 1797, "pos_act_energy_kvar": 165, "rev_act_energy_kvar":
grott        |                   66744, "app_energy_kvar": 238789, "act_energy_kwh": 229223,
grott        |                   "react_energy_kvar": 66910}}
grott        |   - Grott MQTT topic used : energy/growatt

App view:

muppet3000 commented 1 year ago

Thanks for all the info @shocker2 I may not be able to implement this all in one shot, but it would be good to try and get a start on it so you and @gisbertg can test and let me know what it looks like.

I'm thinking I'll start with the act_power_l values as they seem to be explainable.

Are these type of systems "3 Phase" systems and therefore each of the l1, l2, l3 refer to different phases? That would make a lot of sense and I could then label them individually and also provide a sum of them all.

What do you think?

gisbertg commented 1 year ago

Yes exactly as you suspected.

gisbertg commented 1 year ago

https://www.michael-heck.net/wp-content/uploads/2019/08/SDM630-Modbus-V2-14f.pdf

Here you get, I think, all explanations to the transferred values. The manual is in German, interesting for you is page 55, which should also be in English.

https://bg-etech.de/download/manual/SDM630Register.pdf

muppet3000 commented 1 year ago

Hi Guys, I had planned to sit down and work on this tonight, but something's come up and I can't. I'm away for the weekend so I'm unlikely to be able to pick this up again until next week, but it's top of my list.

shocker2 commented 1 year ago

No rush, thank you for your awesome work!

mriksman commented 1 year ago

Hi All - Happy to help provide data if needed. I too have the SDM-ONE (SDM120CTM ).

The integration sees two.

But doesn't have any of the data from the meter.

{"device": "XGD6CAxxxx", "time": "2023-05-20T12:45:59", "buffered": "no", "values": {"datalogserial": "XGD6CAxxxx", "pvserial": "HMG9CAxxx", "voltage_l1": 2461, "voltage_l2": 0, "voltage_l3": 0, "Current_l1": 120, "Current_l2": 0, "Current_l3": 0, "act_power_l1": -27756, "act_power_l2": 0, "act_power_l3": 0, "app_power_l1": 29331, "app_power_l2": 0, "app_power_l3": 0, "react_power_l1": 8444, "react_power_l2": 0, "react_power_l3": 0, "powerfactor_l1": -957, "powerfactor_l2": 0, "powerfactor_l3": 0, "pos_rev_act_power": -27756, "pos_act_power": -27756, "rev_act_power": -27756, "app_power": 29331, "react_power": 8444, "powerfactor": -957, "frequency": 500, "L1-2_voltage": 0, "L2-3_voltage": 0, "L3-1_voltage": 0, "pos_act_energy": 810, "rev_act_energy": 601}}

muppet3000 commented 1 year ago

Hi Guys,

Sorry for not progressing this, I've been away on holiday and busy with family commitments, I'm planning on spending some time in the evenings on this so this is top of my list. I just need to remember where I got to with it!

Thanks for all the really useful feedback and offers for help.

muppet3000 commented 1 year ago

I have just pushed a commit to the main branch that adds the following sensors:

Phase1 Voltage
Phase2 Voltage
Phase3 Voltage
Phase1 Current
Phase2 Current
Phase3 Current
Phase1 Import from Grid Power
Phase2 Import from Grid Power
Phase3 Import from Grid Power
Import from Grid Power
Import from Grid Energy - Total
Export to Grid Energy - Total

Please can people test and give me feedback on what's missing? The obvious one that I think is missing is the PV generation values - does anyone know how to calculate this from the values that are returned?

pmp02 commented 1 year ago

In theory, it should be something like: (voltage_l1 * Current_l1 / powerfactor_l1) + (voltage_l2 * Current_l2 / powerfactor_l2) + (voltage_l2 * Current_l2 / powerfactor_l2)

But from my tests, this doesn't match what the Growatt app shows... so something is missing from the formula.

It would be useful maybe to create entities for all values that are coming from grott, when you have the time, so we can try out different formulas?

muppet3000 commented 1 year ago

Very odd, well I've added the extra values in now, hopefully that gives you something to work with, if there's any extra ones I can plumb through (even with just best-guess names) let me know and you can then have a play.

pmp02 commented 1 year ago

Thanks @muppet3000 ! My take on this would be to have all the extra values in HA, with their actual names from Grott, until we figure it out what they are. But at least we have them for comparison with Growatt App.

shocker2 commented 1 year ago

I have updated from the main repo and I cannot see the import ones.

Below are all the values shown:

muppet3000 commented 1 year ago

@shocker2 that's really weird, are you sure you're running main? On the HACS page for the integration can you check that you've definitely got the main version and not the 1.0.1 release?

shocker2 commented 1 year ago

Well, indeed there is an issue. I have re-downloaded the main (selected main), restarted HA, and checked back it's still 1.0.1.

Not sure why?

muppet3000 commented 1 year ago

That's really weird, it shouldn't do that, that's part of HACS nothing to do with this integration, you've definitely clicked "download" right?

mriksman commented 1 year ago

I oddly had to do it twice, but eventually it worked and I am seeing

muppet3000 commented 1 year ago

I'll try and do a 'proper' versioned release later with all of this stuff on it maybe not close this ticket so that people that are having issues can consume it.

shocker2 commented 1 year ago

Yes, I'm hitting the download button as this triggers the restart notification :) Tried 3 more times and it worked, now I'm on 759caf0. No grid import yet, it might take a few minutes to update.

muppet3000 commented 1 year ago

@shocker2 - Glad you got it installed in the end, let me know if they start appearing or not, sometimes they don't all get published at the same time I believe.

shocker2 commented 1 year ago

Same values, nothing changed:

mriksman commented 1 year ago

I have just pushed a commit to the main branch that adds the following sensors:
* Phase1 Voltage

* Phase2 Voltage

* Phase3 Voltage

* Phase1 Current

* Phase2 Current

* Phase3 Current

* Phase1 Import from Grid Power

* Phase2 Import from Grid Power

* Phase3 Import from Grid Power

* Import from Grid Power

* Import from Grid Energy - Total

* Export to Grid Energy - Total
Please can people test and give me feedback on what's missing? The obvious one that I think is missing is the PV generation values - does anyone know how to calculate this from the values that are returned?

You don't need 'PV Generation Values'. What you have just added is for a 'Smart Meter' - it is measuring grid line voltage/current/power, it has nothing to do with solar generation. With this Smart Meter, you can see if your house (load) is importing power from the grid (solar is less than load) or exporting to grid (solar is more than load). So I now get two devices imported using your Grott - one for the PV Solar Inverter, and one for the Smart Meter

Values look mostly good.

voltage_l2, _l3 seem duplicate of voltage_l1, whilst the values in Grott seem correct (both 0)
powerfactor_l# can be interesting for people to see, and would be worth putting in. Values range from -1 to 1 (although, not in the way you think; -1 and 1 are the same value, it isn't decimal scale)
Your 'Phase # Import from Grid' and 'Import from Grid' - what values are these? They should be the act_power_l# and pos_rev_act_power. I'd change the label to just 'Actual Power', as values can be positive (import) or negative (export).

Not sure why pos_act_power shows a negative value when exporting. I would have thought this should be limited to 0. Anyway, it can be ignored - just use pos_rev_act_power.
You can leave out app_power_l# and react_power_l# - they aren't that important (in my opinion)

muppet3000 commented 1 year ago

@mriksman - Thank you for this awesome feedback, that's exactly what I needed, it's so valuable!!!!!!

I'm working all day today but I'll try and implement all of these (and correct the icons as that's annoying me) tonight.

muppet3000 commented 1 year ago

@mriksman - Thank you again for all your input here!

Feedback on all of the above, hopefully one at a time:

voltage_l2, _l3 seem duplicate of voltage_l1, whilst the values in Grott seem correct (both 0)

You are right, the next commit will fix that

powerfactor_l# can be interesting for people to see, and would be worth putting in. Values range from -1 to 1 (although, not in the way you think; -1 and 1 are the same value, it isn't decimal scale)

I'll add this in, however can you tell me what unit of measurement this is in so I can add it correctly?

Your 'Phase # Import from Grid' and 'Import from Grid' - what values are these? They should be the act_power_l# and pos_rev_act_power. I'd change the label to just 'Actual Power', as values can be positive (import) or negative (export).

Yes your assumption is correct, I'll change the labels so they are "Phase # Actual Power" etc

Not sure why pos_act_power shows a negative value when exporting. I would have thought this should be limited to 0. Anyway, it can be ignored - just use pos_rev_act_power.

I haven't mapped pos_act_power through so that's fine, pos_rev_act_power is mapped as "Import from Grid Power"

You can leave out app_power_l# and react_power_l# - they aren't that important (in my opinion)

I haven't mapped these through yet so maybe I won't bother unless anyone asks.

muppet3000 / homeassistant-grott

energy/meter #7