BUG - Import from grid long-term stats not capturing import used for battery charging

[jasongoff]

Hi,

I use Octopus Agile overnight cheap rates to charge my battery, however the sensors available for import in the Energy dashboard does not capture the kW used for this. In this screenshot you can see that I've imported 1.8kWh to date today (7am) but that the battery charged is 5.3kWh. I've actually imported 7.1kWh overnight, as you can see from the second image (growatt dashboard). I'd like to be able to accurately track the import used alongside the tariffs imported from Octopus, for the purposes of cost tracking. Currently, if I charge my battery, that doesn't work.

[muppet3000]

Hi @jasongoff - I have a few questions:

1. I made a lovely template for 'bugs', when you create a new issue it should present you with a template to complete, did that not happen? It would give me all the information I need to debug this further i.e. what type of system you have
2. Is there a value on the Android app anywhere that would give you this value? If there is can you please send me a screengrab from the app so that I can pinpoint the specific state that needs to be added.

[jasongoff]

Oops! Sorry Chris - I didn't read that properly and just deleted it before I started typing! Copy/pasted below, but to answer your 2nd question sensor.ouctcgp05j_import_from_grid shows the point-in-time values correctly, including the import when I was charging. I can't see any numbers in the mobile app, just the graphs etc. Where would I look do you know?

Describe the bug As above!

To Reproduce Go to Energy Dashboard, view the OUCTCGP05J Import from grid today (load + charging) value. It does not include anything imported to charge the battery.

Growatt Device Type Device(s): SPH5000 Hybrid Inverter

Home Assistant information Home Assistant 2023.1.5 Supervisor 2022.12.1 Operating System 9.4 Frontend 20230110.0 - latest

• Growatt Server API Integration version: The one included in Core - couldn't find a version number

Additional context N/A

[muppet3000]

Thanks @jasongoff - You're forgiven.

Note - if you're still using the version of the integration from Core then I recommend switching to the HACS one so that as/when I fix things you can get them quicker. But it's totally your preference.

I have the same type of inverter and I haven't noticed the same issue, however it doesn't mean that it's not specific to just yourself.

Now, with reference to the specific problem you're having: In order for values to appear on the Energy Dashboard they have to be in "kwh" and an increasing sum e.g. "total lifetime" or "total today" i.e. not an instantaneous value. The particular value you're after has always been a bit of a tricky one to get hold of for hybrid systems, I did have a go at calculating it manually for a while. You'll have to leave it with me, I'll check over the values on the API and come back to you, but it may take a few days (I need to find a gap to look at this again).

The good news is that I have exactly the same system so it shouldn't be too hard to compare against.

[muppet3000]

Can you do me a favour and post this same screenshot but from your phone/app:

It will allow me to compare the values, note that my value for "Imported from grid" (bottom right) doesn't look correct when compared to the graph.

[muppet3000]

I wrote a script a while back for exactly this purpose, but discarded it because I found a value on the API that provided what was needed. Anyway, it looks like they've changed the value of that API call. I've just run the script again now and these were the results:

    *TODAY TOTALS BREAKDOWN*
      Self generation total (batteries & solar - from API) (kwh): 7.2
      Load consumed from solar (kwh): 1.6
      Load consumed from batteries (kwh): 5.6
      Self consumption total (batteries & solar - from API) (kwh): 7.2
      Load consumed from grid (kwh): 4.4
      Total imported from grid (Load + AC charging) (kwh): 4.4
      Load consumption (calculated) (kwh): 11.6
      Load consumption (API) (kwh): 11.6
      Exported (kwh): 0
      Solar battery charge (calculated) (kwh): 1.2
      AC battery charge (calculated) (kwh): 7.5

Note the massive difference in the values between: Total imported from grid (Load + AC charging) (kwh): 4.4 which is from the API call that should have provided the combined value and Load consumption (calculated) (kwh): 11.6 which is calculated with some rather clunky maths (but is far more accurate). I've compared that value to the value on my in home display for my smart meter and it's only 0.3kwh out, so I'd say that's an 'acceptable' tolerance given that it's a calculated value.

Assuming you're seeing the same problem then I think it's time I re-introduced this logic into the Integration and took it for a spin. If you're willing to test it when it's ready you'll need to switch over to the HACS version of the Integration and we can go from there.

I'll keep you posted, NOTE - I'm unlikely to be able to take a look at this before the weekend.

[jasongoff]

Can you do me a favour and post this same screenshot but from your phone/app:

It will allow me to compare the values, note that my value for "Imported from grid" (bottom right) doesn't look correct when compared to the graph.

Bizarrely, I'm in a data deadspot where I work (Swindon town centre!) and the firm's wifi seems to be blocking access to the app! I'll check when I get home.

[jasongoff]

Assuming you're seeing the same problem then I think it's time I re-introduced this logic into the Integration and took it for a spin. If you're willing to test it when it's ready you'll need to switch over to the HACS version of the Integration and we can go from there.

I'll keep you posted, NOTE - I'm unlikely to be able to take a look at this before the weekend.

Hi Chris, yes that looks exactly like the issue I'm seeing. Happy to test, I was planning to switch over to the new version soon anyhow. Thanks!

[muppet3000]

No hurry whatsoever, like I said, I'm unlikely to be able to work on, it issue a fix for this until the weekend at the earliest. The key thing is that it looks like we're both experiencing a similar issue only I had been ignorant to it.

I'll keep you posted on the situation and when I've got a beta release for testing.

Whenever you get the chance to post a screenshot would be helpful just to eyeball that's all.

[jasongoff]

[muppet3000]

Hi @jasongoff - As you can guess I haven't got around to implementing the fix for this yet. I didn't have time over the weekend and I've just spent the evening 'administrating' the hell out of this repo. We now have all the bugs from the Core repo duplicated over here so that I can actually work my way through them. This one is next on my list to fix, hopefully in the next day or so. Sorry for the delay, if it makes you feel any better I have the same system, so I'm experiencing the same bug and it's nagging at me!!

[jasongoff]

Hi @muppet3000, no problem! You're doing this voluntarily alongside your proper* job!

Cheers

Jase

* If yours is anything like mine, tinkering is more fun than the stuff we get paid for :)

[muppet3000]

@jasongoff - If you're feeling adventurous, please try using the 'dev' build of the Integration through HACS (if you can't work out how to do it just let me know here and I'll post screen grabs). I've pushed the logic for calculating the combined 'load + charging' value to be calculated as accurately as we can given the data available. Let me know how you get on. I'm hoping to do a release in the next few days with the logic if you'd prefer to wait for that. There's just 1 more bugfix I want to include in it as well.

[jasongoff]

Hi @muppet3000, All installed so I'll monitor/compare when I charge the battery tonight. Thanks very much :)

[muppet3000]

Hi @muppet3000, All installed so I'll monitor/compare when I charge the battery tonight. Thanks very much :)

In theory if you have an in-home display for your electricity consumption you should see how much you've consumed from the grid so far today. The reading in home-assistant should match up (or be damn close).

Let me know how you get on.

[muppet3000]

FYI - I pushed another tweak to this earlier which you should notice no difference in, it just improves the logic by which this value is calculated to accommodate for when some of the graph data is missing. It still has to make quite sweeping assumptions but it's better than what was there before. One more bug to squash and then I'll do a proper release that contains this.

[jasongoff]

Hi @muppet3000, either I've set it up incorrectly, or I'm not seeing the changes. I installed DEV release in HACS, but I'm not seeing the intended results. Battery charged overnight for 6kwh, but import+load showing as 1.5kwh rather 7.5.

[muppet3000]

Did you remember to restart Home Assistant after installing the Dev version? (If you go to the HACS dashboard in HA it normally has a big red warning if you haven't restarted since the update) If you did then I'm not sure what the issue could be. If a restart of HA doesn't fix it would you consider sharing access to your system with me so I can help debug? There are steps on how to share them on the FAQ page for this repo.

[jasongoff]

Hi, yes did a restart. Sent you a personal email. Cheers!

Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: Chris Straffon @.> Sent: Friday, February 3, 2023 7:12:15 AM To: muppet3000/homeassistant-growatt_server_api @.> Cc: Jason Goff @.>; Mention @.> Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)

Did you remember to restart Home Assistant after installing the Dev version? (If you go to the HACS dashboard in HA it normally has a big red warning if you haven't restarted since the update) If you did then I'm not sure what the issue could be. If a restart of HA doesn't fix it would you consider sharing access to your system with me so I can help debug? There are steps on how to share them on the FAQ page for this repo.

— Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1415191889, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA4OOQ6CIE2DUYCWX4LWVSVU7ANCNFSM6AAAAAAT6WQC4Y. You are receiving this because you were mentioned.Message ID: @.***>

[muppet3000]

Hi @jasongoff I think you've misunderstood, I don't need access to your HA installation, I'd like to use your credentials for Growatt on my Home Assistant test instance to debug the issue. You can find steps on how to do that here: https://github.com/muppet3000/homeassistant-growatt_server_api/blob/dev/FAQ.md#how-do-i-share-my-credentials-with-you-for-testingadding-new-features-for-my-system-type

[jasongoff]

Hi, done! email should be coming your way from Growatt. Let me know if not.

---

From: Chris Straffon @.> Sent: 03 February 2023 8:07 AM To: muppet3000/homeassistant-growatt_server_api @.> Cc: Jason Goff @.>; Mention @.> Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)

Hi @jasongoffhttps://github.com/jasongoff I think you've misunderstood, I don't need access to your HA installation, I'd like to use your credentials for Growatt on my Home Assistant test instance to debug the issue. You can find steps on how to do that here: https://github.com/muppet3000/homeassistant-growatt_server_api/blob/dev/FAQ.md#how-do-i-share-my-credentials-with-you-for-testingadding-new-features-for-my-system-type

— Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1415292642, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA4DC4CAC42RJVX6UXDWVS4CXANCNFSM6AAAAAAT6WQC4Y. You are receiving this because you were mentioned.Message ID: @.***>

[muppet3000]

Thanks I've got it, I'll try and take a look. The maths doesn't quite add up, I agree. I've got a little helper app that I wrote to debug this sort of stuff so I'll run it through that tonight and see what comes out!

[muppet3000]

Sent you an email. I accidentally sent it here too but I've deleted it from here so you probably have the same email twice.

[jasongoff]

Hi @muppet3000 ,

Now I've actually got the dev branch installed properly, I've checked this morning and it is indeed working out the numbers correctly! Thanks very much, and apologies for the confusion!

Cheers

Jase

[muppet3000]

Awesome, thanks for the update, I was beginning to think my logic was twitchy!!

I've got 1 more bug to squash and then I'll wrap it up in a formal release.

[jasongoff]

Hi @muppet3000

I downloaded the latest update and my mobile account has locked out. Should i have changed something in the setup?

[muppet3000]

I recommend taking a look over here: https://github.com/muppet3000/homeassistant-growatt_server_api/issues/15

There have been various issues with the Growatt servers today and they seem to have started locking accounts for some reason.

My suggestion from over on that other thread is to log into the growatt website (server.growatt.com) and create yourself a different user for use with Home Assistant (and potentially one for your phone as well) to work around this issue. i.e. follow the same steps you did to create a user for me, but just create multiple for yourself so that you don't get locked out on your phone again.

I am also re-adding the ability to specify "server.growatt.com" back into the integration (available on Dev now) as that endpoint is working again and may be more stable.

[jasongoff]

Thanks Chris,

I'd already thought about creating separate accounts to preserve the master one, so will get that sorted tomorrow.

Cheers

Jase

Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: Chris Straffon @.> Sent: Tuesday, February 7, 2023 9:34:43 PM To: muppet3000/homeassistant-growatt_server_api @.> Cc: Jason Goff @.>; Mention @.> Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)

I recommend taking a look over here: #15https://github.com/muppet3000/homeassistant-growatt_server_api/issues/15

There have been various issues with the Growatt servers today and they seem to have started locking accounts for some reason.

My suggestion from over on that other thread is to log into the growatt website (server.growatt.com) and create yourself a different user for use with Home Assistant (and potentially one for your phone as well) to work around this issue. i.e. follow the same steps you did to create a user for me, but just create multiple for yourself so that you don't get locked out on your phone again.

I am also re-adding the ability to specify "server.growatt.com" back into the integration (available on Dev now) as that endpoint is working again and may be more stable.

— Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1421484204, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA2O3SAGJG3OZQWBOZ3WWK5XHANCNFSM6AAAAAAT6WQC4Y. You are receiving this because you were mentioned.Message ID: @.***>

[muppet3000]

No problem, it seems to have happened to a load of people today.

[jasongoff]

Hi,

I can't see where to update my account details in the HA integration.

Sent from Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: Chris Straffon @.> Sent: Tuesday, February 7, 2023 9:34:43 PM To: muppet3000/homeassistant-growatt_server_api @.> Cc: Jason Goff @.>; Mention @.> Subject: Re: [muppet3000/homeassistant-growatt_server_api] BUG - Import from grid long-term stats not capturing import used for battery charging (Issue #7)

I recommend taking a look over here: #15https://github.com/muppet3000/homeassistant-growatt_server_api/issues/15

There have been various issues with the Growatt servers today and they seem to have started locking accounts for some reason.

My suggestion from over on that other thread is to log into the growatt website (server.growatt.com) and create yourself a different user for use with Home Assistant (and potentially one for your phone as well) to work around this issue. i.e. follow the same steps you did to create a user for me, but just create multiple for yourself so that you don't get locked out on your phone again.

I am also re-adding the ability to specify "server.growatt.com" back into the integration (available on Dev now) as that endpoint is working again and may be more stable.

— Reply to this email directly, view it on GitHubhttps://github.com/muppet3000/homeassistant-growatt_server_api/issues/7#issuecomment-1421484204, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFTA2O3SAGJG3OZQWBOZ3WWK5XHANCNFSM6AAAAAAT6WQC4Y. You are receiving this because you were mentioned.Message ID: @.***>

[muppet3000]

You have to delete the integration and then re-add it, one of the weird quirks of HA, it's actually really difficult to write the logic to allow people to change their username/password, just easier to delete and re-add.

Since it's the same instance you're pointing to it'll still have all your previous data.

From my phone but you get the idea...

[muppet3000]

Closing - fixed in https://github.com/muppet3000/homeassistant-growatt_server_api/releases/tag/v1.0.2