MickMake / GoSungrow

GoLang implementation to access the iSolarCloud API updated by SunGrow inverters.
https://mickmake.com/
GNU General Public License v2.0
148 stars 42 forks source link

Error 'er_invalid_appkey' / 'Request is not encrypted' #101

Open rob3r7 opened 7 months ago

rob3r7 commented 7 months ago

Since tonight I get with https://gateway.isolarcloud.eu in HA the following error:

ERROR: appkey is incorrect 'er_invalid_appkey

Checking with https://portaleu.isolarcloud.com/ I realized that the appkey could have changed to B0455FBE7AA0328DB57B59AA729F05D8 (at least I find this key when searching for the term appkey) .

When doing a direct request at /v1/userService/login at least I don't get any more an invalid_appkey error but now an Request is not encrypted error.

When looking at the source of https://portaleu.isolarcloud.com/#/dashboard there is the following function:

e.data.set("appkey", a.a.encryptHex(e.data.get("appkey"), h))

Did Sungrow changed the API access? How to deal with this change?

Razor094 commented 7 months ago

I'm using the ModbusTCP2MQTT now, with my Sungrow SG5.0RT inverter, and it's working fine.... If you can't wait the fix, try it... https://github.com/MatterVN/ModbusTCP2MQTT

for the time being I am using this addon with direct access to SH6.0RT, which works perfectly fine. I hope we will have a fixed for his asap.

I am also using it since yesterday with a SG8.0RT and it seems it's working (even though I dont have much sun at this time of the year where I am :D )

Can you Share your Settings? I try to use this addon with my SG10RT but don’t get any data...

grf692 commented 7 months ago

First, I stopped and uninstalled the GoSungrow addon.

Then, if not already done, you need to install Mosquitto broker MQTT addon and make sure it works and that you have an appropriate user you can use with it. At this point I think I restarted HA so that MQTT is detected and appears under settings / devices / integrations.

image

Then you install the ModbusTCP2MQTT addon. I don't think I've done anything fancy but just the default config.

image

Finally I configured my Energy dashboard to use the new entity.

image

plumpy80 commented 7 months ago

I'm using a Winet-S dongle, and Wifi network, so this config works for me:

image

For me, the MQTT was installed, because of Zigbee2MQTT. After one restart the ModbusTCP2MQTT start to work without any MQTT specific settings.

AZKhalil commented 7 months ago

this worked with SH6.0RT sungrow connection was giving me random error messages.

image

plumpy80 commented 7 months ago

If you have WiNet-S dongle, and using it with Wifi, you need to use the HTTP connection with the 8082 port. Log_level: DEBUG? Why? Standard setting is the "INFO"

grf692 commented 7 months ago

I have WiNet-S dongle and I am using it with Wifi, but it seems it working with the default Sungrow and port settings (as in the screenshot from my post above) - I might be wrong but I do think it works :D

plumpy80 commented 7 months ago

BTW, just try every combination.... πŸ˜„ πŸ˜„

Maartenf2 commented 7 months ago

Stupid question: How do I find my inverter IP?

plumpy80 commented 7 months ago

I find it with the Nmap app on Windows. Juts listed all used IP address, and I have only one with 8082 port. This is my Inverters IP. This shit using a hided IP adress... Very annoying....

threepoints85 commented 7 months ago

I installed the ModbusTCP2MQTT and get all the Infos from the Inverter. However, I don't see any data from the storage displayed. Is there a solution for this?

quim123 commented 7 months ago

Here is another working Python example of how to use query the new Sungrow API. Sadly, I forgot to refresh the page while working and didn't see the post of @rob3r7 ...

The scope of the code below is very similar to his contribution, with some extra additions:

* I have added a call to log in to the portal (this requires a different RSA key)

* My x-limit-obj adds the RSA encrypted user id, which is required for some calls, as also mentioned by @0SkillAllLuck

You will see that the post method has a isFormData flag. The code associated with that flag being true is reverse engineered based on what I was able to make out of minified Sungrow encryption code. I do not know a call for which the flag should be set to true, so in that particular case there may be some issues...

import base64
import string
import random
from typing import Optional
import time
import json

import requests
from cryptography.hazmat.primitives import serialization, asymmetric, padding
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.backends import default_backend

# TODO: Getting these values directly from the files by the Sungrow API is better than hardcoding them...
LOGIN_RSA_PUBLIC_KEY: asymmetric.rsa.RSAPublicKey = serialization.load_pem_public_key(b"-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDJRGV7eyd9peLPOIqFg3oionWqpmrjVik2wyJzWqv8it3yAvo/o4OR40ybrZPHq526k6ngvqHOCNJvhrN7wXNUEIT+PXyLuwfWP04I4EDBS3Bn3LcTMAnGVoIka0f5O6lo3I0YtPWwnyhcQhrHWuTietGC0CNwueI11Juq8NV2nwIDAQAB\n-----END PUBLIC KEY-----")
APP_RSA_PUBLIC_KEY: asymmetric.rsa.RSAPublicKey   = serialization.load_pem_public_key(bytes("-----BEGIN PUBLIC KEY-----\n" + "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkecphb6vgsBx4LJknKKes-eyj7-RKQ3fikF5B67EObZ3t4moFZyMGuuJPiadYdaxvRqtxyblIlVM7omAasROtKRhtgKwwRxo2a6878qBhTgUVlsqugpI_7ZC9RmO2Rpmr8WzDeAapGANfHN5bVr7G7GYGwIrjvyxMrAVit_oM4wIDAQAB".replace("-", "+").replace("_", "/") + "\n-----END PUBLIC KEY-----",  'utf8'))
ACCESS_KEY = "9grzgbmxdsp3arfmmgq347xjbza4ysps"
APP_KEY = "B0455FBE7AA0328DB57B59AA729F05D8"

def encrypt_rsa(value: str, key: asymmetric.rsa.RSAPublicKey) -> str:
    # Encrypt the value
    encrypted = key.encrypt(
        value.encode(),
        asymmetric.padding.PKCS1v15(),
    )
    return base64.b64encode(encrypted).decode()

def encrypt_aes(data: str, key: str):
    key_bytes = key.encode('utf-8')
    data_bytes = data.encode('utf-8')

    # Ensure the key is 16 bytes (128 bits)
    if len(key_bytes) != 16:
        raise ValueError("Key must be 16 characters long")

    cipher = Cipher(algorithms.AES(key_bytes), modes.ECB(), backend=default_backend())
    encryptor = cipher.encryptor()
    padder = padding.PKCS7(algorithms.AES.block_size).padder()
    padded_data = padder.update(data_bytes) + padder.finalize()
    encrypted_data = encryptor.update(padded_data) + encryptor.finalize()
    return encrypted_data.hex()

def decrypt_aes(data: str, key: str):
    key_bytes = key.encode('utf-8')

    # Ensure the key is 16 bytes (128 bits)
    if len(key_bytes) != 16:
        raise ValueError("Key must be 16 characters long")

    encrypted_data = bytes.fromhex(data)
    cipher = Cipher(algorithms.AES(key_bytes), modes.ECB(), backend=default_backend())
    decryptor = cipher.decryptor()
    decrypted_padded_data = decryptor.update(encrypted_data) + decryptor.finalize()
    unpadder = padding.PKCS7(algorithms.AES.block_size).unpadder()
    decrypted_data = unpadder.update(decrypted_padded_data) + unpadder.finalize()
    return decrypted_data.decode('utf-8')

def generate_random_word(length: int):
    char_pool = string.ascii_letters + string.digits
    random_word = ''.join(random.choice(char_pool) for _ in range(length))
    return random_word

class SungrowScraper:
    def __init__(self, username: str, password: str):
        self.baseUrl = "https://www.isolarcloud.com"
        # TODO: Set the gateway during the login procedure
        self.gatewayUrl = "https://gateway.isolarcloud.eu"
        self.username = username
        self.password = password
        self.session: "requests.Session" = requests.session()
        self.userToken: "str|None" = None

    def login(self):
        self.session = requests.session()
        resp = self.session.post(
            f"{self.baseUrl}/userLoginAction_login",
            data={
                "userAcct": self.username,
                "userPswd": encrypt_rsa(self.password, LOGIN_RSA_PUBLIC_KEY),
            },
            headers={
                "_isMd5": "1"
            },
            timeout=60,
        )
        self.userToken = resp.json()["user_token"]
        return self.userToken

    def post(self, relativeUrl: str, jsn: "Optional[dict]"=None, isFormData=False):
        userToken = self.userToken if self.userToken is not None else self.login()
        jsn = dict(jsn) if jsn is not None else {}
        nonce = generate_random_word(32)
        # TODO: Sungrow also adjusts for time difference between server and client
        # This is probably not a must though. The relevant call is:
        # https://gateway.isolarcloud.eu/v1/timestamp
        unixTimeMs = int(time.time() * 1000)
        jsn["api_key_param"] = {"timestamp": unixTimeMs, "nonce": nonce}
        randomKey = "web" + generate_random_word(13)
        userToken = self.userToken
        userId = userToken.split('_')[0]
        jsn["appkey"] = APP_KEY
        if "token" not in jsn:
            jsn["token"] = userToken
        jsn["sys_code"] = 200
        data: "dict|str"
        if isFormData:
            jsn["api_key_param"] = encrypt_aes(json.dumps(jsn["api_key_param"]), randomKey)
            jsn["appkey"] = encrypt_aes(jsn["appkey"], randomKey)
            jsn["token"] = encrypt_aes(jsn["token"], randomKey)
            data = jsn
        else:
            data = encrypt_aes(json.dumps(jsn, separators=(",", ":")), randomKey)
        resp = self.session.post(
            f"{self.gatewayUrl}{relativeUrl}",
            data=data,
            headers={
                "x-access-key": ACCESS_KEY,
                "x-random-secret-key": encrypt_rsa(randomKey, APP_RSA_PUBLIC_KEY),
                "x-limit-obj": encrypt_rsa(userId, APP_RSA_PUBLIC_KEY),
                "content-type": "application/json;charset=UTF-8"
            }
        )
        return decrypt_aes(resp.text, randomKey)

s = SungrowScraper("MY_USERNAME", "MY_PASSWORD")
resp = s.post(
    "/v1/powerStationService/getPsListNova",
    jsn={
        "share_type_list": ["0", "1", "2"]
    }
)
print(resp)

Using Python it works!! Thank you! I modified the resp to point to "/v1/commonService/queryMutiPointDataList".

threepoints85 commented 7 months ago

@quim123 Can you explain what to do?

triamazikamno commented 7 months ago

I've made a very dirty and quick Go port of those python implementations: https://github.com/triamazikamno/GoSungrow/commit/5bb253cfc141cd537f040e73eb2fa3ebc118672d Feel free to build from the encryption branch of my fork. I've done very limited testing, but so far it's working fine for me.

Note: you need to change appkey in your config.json to B0455FBE7AA0328DB57B59AA729F05D8

metawops commented 7 months ago

Awesome! The ModbusTCP2MQTT Add-on works fine for me, too. No need for this Golang based integration anymore. Like others above I, too, have a SG6.0RT (that's the cipher 0 there, not the letter O, btw.) and I needed a while until I found out the IP of the inverter. Key to success was this nmap command:

nmap -p 8082 --open -T4 -Pn 192.168.0.0/24

that scans my whole 192.168.0.* local network to find that IP where port 8082 is open. And it found this:

Nmap scan report for espressif.fritz.box (192.168.0.104)
Host is up (0.010s latency).

PORT     STATE SERVICE
8082/tcp open  blackice-alerts

Nmap done: 256 IP addresses (256 hosts up) scanned in 13.79 seconds

So they probably use a simple & cheap ESP32 (or even cheaper ESP8266) chip from Espressif!! And 192.168.0.104 is the IP I needed to put into the integration's config. βœ…

Inverter_host: 192.168.0.104
Inverter_port: 502
Inverter_model: SG6.0RT
Smart_meter: true
Connection: HTTP
Scan_level: ALL
Scan_interval: 30
Scan_timeout: 5
Log_level: INFO

In my Fritz!Box router I changed the name espressif to espressif-sungrow-inverter and made it use the same IPv4 address everytime. πŸ˜‰

bobosch commented 7 months ago

Here is another working Python example of how to use query the new Sungrow API. Sadly, I forgot to refresh the page while working and didn't see the post of @rob3r7 ...

@Pistro Maybe you can create a repository with your Python version?

monojk commented 7 months ago

Does the ModbusTCP2MQTT also provide historical data like of previous month (e.g. 15 minutes interval data) or does it just provide the current data?

luongdaniel commented 7 months ago

Does the ModbusTCP2MQTT also provide historical data like of previous month (e.g. 15 minutes interval data) or does it just provide the current data?

Only current data.

It also doesn't seem to provide the grid import and export (kWh) data so it works for me as a temporary solution just to view live production and consumption but it's not a perfect replacement for GoSungrow for me.

AZKhalil commented 7 months ago

If you have WiNet-S dongle, and using it with Wifi, you need to use the HTTP connection with the 8082 port. Log_level: DEBUG? Why? Standard setting is the "INFO"

I have already changed the log level to info, i had issues with sungrow connection type so needed to see the logs, my module has wifi but i am only using the LAN.

zrahash commented 7 months ago

anyone able to get the consumption reading with sungrow hybrid inverter with WiNet-S dongle ? The generation working for me.

luongdaniel commented 7 months ago

anyone able to get the consumption reading with sungrow hybrid inverter with WiNet-S dongle ? The generation working for me.

I was able to get consumption by toggling off the 'smart meter' config parameter.

zrahash commented 7 months ago

I have enabled it and tried to rebuild again but I only see sensor.solar_inverter_daily generation . any idea ?

turnipy commented 7 months ago

Awesome! The ModbusTCP2MQTT Add-on works fine for me, too. No need for this Golang based integration anymore. Like others above I, too, have a SG6.0RT (that's the cipher 0 there, not the letter O, btw.) and I needed a while until I found out the IP of the inverter. Key to success was this nmap command:

nmap -p 8082 --open -T4 -Pn 192.168.0.0/24

that scans my whole 192.168.0.* local network to find that IP where port 8082 is open. And it found this:

Nmap scan report for espressif.fritz.box (192.168.0.104)
Host is up (0.010s latency).

PORT     STATE SERVICE
8082/tcp open  blackice-alerts

Nmap done: 256 IP addresses (256 hosts up) scanned in 13.79 seconds

So they probably use a simple & cheap ESP32 (or even cheaper ESP8266) chip from Espressif!! And 192.168.0.104 is the IP I needed to put into the integration's config. βœ…

Inverter_host: 192.168.0.104
Inverter_port: 502
Inverter_model: SG6.0RT
Smart_meter: true
Connection: HTTP
Scan_level: ALL
Scan_interval: 30
Scan_timeout: 5
Log_level: INFO

In my Fritz!Box router I changed the name espressif to espressif-sungrow-inverter and made it use the same IPv4 address everytime. πŸ˜‰

Funny, I wish I saw this before working it out myself but what you have described is exactly what I came back to post here so others can get it working. As you discovered, the trick was finding out the IP of the comms device, which is tagged by expressif in my router DHCP server. Then in the config, set to http and the right model, mine is SG10RT, then all working. I will set the IP to reserved later so it doesn't change, otherwise all now fine. For those who don't know (I didn't either), if you open the IP in a browser, you also see a sungrow local portal. Note the login to get admin (if it hasnt changed from default) is U: admin P: pw8888 and non admin is U: user P: pw1111. When you first login, it takes me to a setup wizard - just close that and then you are in the portal.

Note though, for my unit I had to set Inverter port to: 80. 502 didnt work. Otherwise same as above (aside from different IP)

Paraphraser commented 7 months ago

A lot of people are following this issue in the hope than an official update will be released. Although the side-track into ModbusTCP2MQTT is fascinating (and I'll probably try it on my own WiFi-connected SG5.0RS sometime), I think there's a small risk that the original er_invalid_appkey problem is being buried.

So, just for now, I'd like to haul the discussion back to the original problem.

Earlier, @triamazikamno wrote:

I've made a very dirty and quick Go port of those python implementations: triamazikamno@5bb253c Feel free to build from the encryption branch of my fork. I've done very limited testing, but so far it's working fine for me.

Note: you need to change appkey in your config.json to B0455FBE7AA0328DB57B59AA729F05D8

I can confirm that this is working for me too. The earlier post was a little short on the how-to so I've updated this gist on compiling GoSungrow to explain how I applied that to my own situation.

The way I use GoSungrow is from a cron job that fires once a day to fetch "yesterday's" data which I munge into an SQLite database. So, for me, just being able to recompile with the excellent fix from @triamazikamno has gotten me going again. I was able to download all missing data since everything turned to custard.

I am not a big HomeAssistant user. I have it running but only for tinkering and I can't claim anything more than the shallowest of shallow knowledge of the whole HA ecosystem.

I do, however, have a reasonable amount of Docker knowledge so I was able to apply that to add the recompiled GoSungrow binary to the base image from MickMake that was already on my HA system, and get it to work.

For all I know a HomeAssistant guru would take one look at my approach and say, "oh, that's doing it the hard way - you just need to …". If anyone reading this is such a guru and is able to enlighten me as to the True Path, I'd really welcome that because this kind of problem (of having a fix in advance of a formal release) is bound to come up again.

In the meantime, I've added my hack to the same gist in case the same approach is useful for anyone else.

Crisse771 commented 7 months ago

I agree with Paraphraser. Is the ModbusTCP2MQTT solution capable of, for instance, reading information related to a battery connected to the inverter? i.e. "Battery temperature"

Chears !

KLucky13 commented 7 months ago

I was able to apply that to add the recompiled GoSungrow binary to the base image from MickMake that was already on my HA system, and get it to work.

For all I know a HomeAssistant guru would take one look at my approach and say, "oh, that's

Thanks for you time you put into this. Even this for me as a basic user is chinese, so I hope the official fix will follow soon. Maybe with the use of GPT I can try to implement this fix. But I'm afraid that this will never be officially fixed since we still have no statement of @MickMake that he will be looking into it.

Would there be an option that somebody could "just" fix the issue and add it as a new add-on to HA? So for basic users like me it can be installed like it is just a new add-on, instead of thinkering in the existing code

SCoe2k commented 7 months ago

Here we go. A first minimal MVP. The api_key_param is updated with each request before encryption.

import json
import random
import string
from base64 import b64decode, b64encode, urlsafe_b64decode
from datetime import datetime

import requests
from Crypto.Cipher import AES, PKCS1_v1_5
from Crypto.PublicKey import RSA
from Crypto.Util.Padding import pad, unpad

def encrypt_hex(data_str, key):
    cipher = AES.new(key.encode("UTF-8"), AES.MODE_ECB)
    date_byte = cipher.encrypt(pad(data_str.encode("UTF-8"), 16))
    return date_byte.hex()

def decrypt_hex(data_hex_str, key):
    cipher = AES.new(key.encode("UTF-8"), AES.MODE_ECB)
    text = unpad(cipher.decrypt(bytes.fromhex(data_hex_str)), 16).decode("UTF-8")
    return json.loads(text)

public_key = RSA.import_key(
    urlsafe_b64decode(
        "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkecphb6vgsBx4LJknKKes-eyj7-RKQ3fikF5B67EObZ3t4moFZyMGuuJPiadYdaxvRqtxyblIlVM7omAasROtKRhtgKwwRxo2a6878qBhTgUVlsqugpI_7ZC9RmO2Rpmr8WzDeAapGANfHN5bVr7G7GYGwIrjvyxMrAVit_oM4wIDAQAB"
    )
)
cipher = PKCS1_v1_5.new(public_key)

def encrypt_RSA(data_str):
    ciphertext = cipher.encrypt(data_str.encode("UTF-8"))
    return b64encode(ciphertext).decode("UTF-8")

def random_word(length):
    characters = string.ascii_lowercase + string.ascii_uppercase + string.digits
    random_word = "".join(random.choice(characters) for _ in range(length))
    return random_word

def get_data(url, data):
    random_key = "web" + random_word(13)
    data["api_key_param"] = {
        "timestamp": int(datetime.now().timestamp() * 1000),
        "nonce": random_word(32),
    }
    data["appkey"] = "B0455FBE7AA0328DB57B59AA729F05D8"
    data_str = json.dumps(data, separators=(",", ":"))
    data_hex = encrypt_hex(data_str, random_key)
    headers = {
        "content-type": "application/json;charset=UTF-8",
        "sys_code": "200",
        "x-access-key": "9grzgbmxdsp3arfmmgq347xjbza4ysps",
    }
    headers["x-random-secret-key"] = encrypt_RSA(random_key)

    response = requests.post(url, data=data_hex, headers=headers)
    return decrypt_hex(response.text, random_key)

token = get_data(
    "https://gateway.isolarcloud.eu/v1/userService/login",
    {
        "user_account": "XXXXX@XXXXX.ch",
        "user_password": "XXXXXXXX",
    },
)["result_data"]["token"]

get_data(
    "https://gateway.isolarcloud.eu/v1/commonService/queryMutiPointDataList",
    {
        "ps_key": "XXXXXXX_14_1_2",
        "points": "p13003",
        "start_time_stamp": "20231108000000",
        "end_time_stamp": "20231109000000",
        "token": token,
    },
)

@0SkillAllLuck: GrΓΌsse nach Bern!

Sorry for the question. How/Where to can I implement your fix in HA?

threepoints85 commented 7 months ago

In the meantime, I've added my hack to the same gist in case the same approach is useful for anyone else.

Thanks. Unfortunately i don't understand anything :-) Because of the language barrier and Im not very familiar with the topic.

Is there an easier way to fix it?

metawops commented 7 months ago

Does the ModbusTCP2MQTT also provide historical data like of previous month (e.g. 15 minutes interval data) or does it just provide the current data?

If you're using Home Assistant you can install a long-time database add-on like InfluxDB that records all data of all sensors for (almost) ever ... πŸ˜‰

piyushbhatia10 commented 7 months ago

Since tonight I get with https://gateway.isolarcloud.eu in HA the following error:

ERROR: appkey is incorrect 'er_invalid_appkey

Checking with https://portaleu.isolarcloud.com/ I realized that the appkey could have changed to B0455FBE7AA0328DB57B59AA729F05D8 (at least I find this key when searching for the term appkey) .

When doing a direct request at /v1/userService/login at least I don't get any more an invalid_appkey error but now an Request is not encrypted error.

When looking at the source of https://portaleu.isolarcloud.com/#/dashboard there is the following function:

e.data.set("appkey", a.a.encryptHex(e.data.get("appkey"), h))

Did Sungrow changed the API access? How to deal with this change?

Where did you look to find the appkey on that link?

jarg commented 7 months ago

I'm using the ModbusTCP2MQTT now, with my Sungrow SG5.0RT inverter, and it's working fine.... If you can't wait the fix, try it...

https://github.com/MatterVN/ModbusTCP2MQTT

In my case I have 3 inverters connected in series and with the isolar wifi dongle. I would have to wire the inverters. the wifi dongle does not allow modbus or anything like that.

I am waiting for a solution for this addon.

Thanks to all.

triamazikamno commented 7 months ago

In the meantime, I've added my hack to the same gist in case the same approach is useful for anyone else.

Thank you @Paraphraser for that helpful gist! I have no experience with HomeAssistant, but I've managed to build, push and test docker images for the addon on all platforms, here is a simplified way to do a hack now:

  1. connect to the HA instance over ssh
  2. run:
    old_image=`docker image list --format 'table {{.Repository}}:{{.Tag}}' | grep gosungrow | grep -vE 'triamazikamno|backup' | head -1`;
    new_image=`echo $old_image | awk -F/ '{print"triamazikamno/"$2}'`;
    docker tag $old_image ${old_image}-backup;
    docker pull $new_image;
    docker tag $new_image $old_image;
  3. go to the HomeAssistant GUI, Settings, Add-ons, GoSungrow, Configuration tab and change sungrow_appkey to B0455FBE7AA0328DB57B59AA729F05D8

Restart GoSungrow when it prompts.

Thibaut1976 commented 7 months ago

_old_image=docker ps -a | grep gosungrow | head -1 | awk '{print$2}'; new_image=echo $old_image | awk -F/ '{print"triamazikamno/"$2}'; docker tag $old_image ${old_image}-backup; docker pull $new_image; docker tag $new_image $oldimage;

When I executed on SSH : I get command not found, any idea how to resolve ?

Paraphraser commented 6 months ago

@Thibaut1976

When I executed on SSH : I get command not found, any idea how to resolve ?

A screen shot would help to know exactly which command wasn't found. However, I think I've figured out what might be going on so I'm going to make a guess and see if that helps.

In my gist, I made an implicit assumption. I assumed each of us would be working on a Home Assistant system where the GoSungrow add-on was broken.

On my system at least, that meant the add-on was not running. To put that another way, if I went to the GoSungrow add-on's "Info" tab, I saw "START", rather than "STOP" and "RESTART".

The commands provided by @triamazikamno rely on docker ps. I think that strategy will only work if the container is running.

I've replicated what I believe is the intention. Here it is, step by step. If you have any trouble, please tell me what step you got up to and consider including a screen shot:

  1. The "Advanced SSH & Web Terminal" add-on must be installed. In that add-on's:

    • "Info" tab: "Protection mode" needs to be turned off.
    • "Configuration" tab: you need to set a username and password. My username is "hassio". It turns up in the ssh command below.
    • "Info" tab: if you changed any settings, click "RESTART". If you don't see "RESTART" then click "START".
  2. From your support host (Linux, macOS, Windows), connect to the HA instance:

    $ ssh hassio@homeassistant.local

    The prompt you get is:

    ~ #

    which indicates you are running as root and your working directory is root's home directory. In what follows, wherever you see a line starting with "#", it means "copy/paste everything except the "#" and press return".

  3. You should be able to see the existing (broken) image:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG         IMAGE ID       CREATED        SIZE
    ba22da74/amd64-addon-gosungrow                    3.0.7       2f8714749ba2   3 months ago   161MB

    Your output may be slightly different but you should at least be able to identify a line with "gosungrow" and version 3.0.7.

  4. Construct the required variables. I use slightly different syntax but it achieves the same goal:

    # old_image=$(docker images | grep gosungrow | awk '{print $1 ":" $2}')
    # echo $old_image
    ba22da74/amd64-addon-gosungrow:3.0.7
    
    # new_image=$(echo $old_image | awk -F/ '{print"triamazikamno/"$2}')
    # echo $new_image
    triamazikamno/amd64-addon-gosungrow:3.0.7

    You should get something sensible in response to the echo commands. Your exact output may differ but you should be able to see that the earlier output from docker images has turned up in the first echo command, and that the first part of the image name (ie ba22da74) has been replaced with triamazikamno.

  5. Retag the old image to keep it hanging around:

    # docker tag $old_image ${old_image}-backup
  6. You can confirm that that has worked by re-running the images command:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG            IMAGE ID       CREATED        SIZE
    ba22da74/amd64-addon-gosungrow                    3.0.7          2f8714749ba2   3 months ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7-backup   2f8714749ba2   3 months ago   161MB

    Two repository+tag combinations pointing to the same Image ID.

  7. Pull the new image from DockerHub:

    # docker pull $new_image
    3.0.7: Pulling from triamazikamno/amd64-addon-gosungrow
    659d66d51139: Already exists 
    7c0ba91aad39: Pull complete 
    fb2a01b55562: Pull complete 
    4425acca1925: Pull complete 
    d50c5eb93aa0: Pull complete 
    adde5526d152: Pull complete 
    Digest: sha256:216c20966785878ccae85b48e45f31fc5e38295f04589d0b2377a7c8b564c867
    Status: Downloaded newer image for triamazikamno/amd64-addon-gosungrow:3.0.7
    docker.io/triamazikamno/amd64-addon-gosungrow:3.0.7

    Again, the actual details may vary but, once again, images will confirm the result:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG            IMAGE ID       CREATED        SIZE
    triamazikamno/amd64-addon-gosungrow               3.0.7          f2cbc9418287   11 hours ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7          2f8714749ba2   3 months ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7-backup   2f8714749ba2   3 months ago   161MB

    Three repo+tag combo, two distinct images.

  8. Now we change the middle tag to point to the new image:

    # docker tag $new_image $old_image

    And confirm that again with images:

    # docker images | grep -e REPOSITORY -e gosungrow
    REPOSITORY                                        TAG            IMAGE ID       CREATED        SIZE
    ba22da74/amd64-addon-gosungrow                    3.0.7          f2cbc9418287   11 hours ago   161MB
    triamazikamno/amd64-addon-gosungrow               3.0.7          f2cbc9418287   11 hours ago   161MB
    ba22da74/amd64-addon-gosungrow                    3.0.7-backup   2f8714749ba2   3 months ago   161MB
  9. Go back to the GUI. Settings. Add-ons. GoSungrow:

    • "Configuration" tab. Change the sungrow_appkey to:

      B0455FBE7AA0328DB57B59AA729F05D8
    • "Info" tab. Click "START".

In writing the above, I repeated the steps a few times to make sure I'd captured everything. On one occasion, HA chucked a wobbly and restarted after I clicked the "START" button. It came back OK and GoSungrow was running. I just mention it in case the same thing happens to you.

milessw commented 6 months ago

Changed the App_Key and getting this now [03:38:56] INFO: Login to iSolarCloud using gateway https://augateway.isolarcloud.com ... Error: unknown error 'Request is not encrypted' Usage: GoSungrow api login [flags]

Examples: GoSungrow api login

Flags: Use "GoSungrow help flags" for more info.

Additional help topics:

ERROR: unknown error 'Request is not encrypted' s6-rc: info: service legacy-services: stopping s6-rc: info: service legacy-services successfully stopped s6-rc: info: service legacy-cont-init: stopping s6-rc: info: service legacy-cont-init successfully stopped s6-rc: info: service fix-attrs: stopping s6-rc: info: service fix-attrs successfully stopped s6-rc: info: service s6rc-oneshot-runner: stopping s6-rc: info: service s6rc-oneshot-runner successfully stopped

Been a week, Author is MIA. So what's going on?

Paraphraser commented 6 months ago

@milessw

The problem is that, as well as changing the App Key, you also need to fake HA into using an updated image.

Please follow the steps in this comment.

eXplague commented 6 months ago

In the meantime, I've added my hack to the same gist in case the same approach is useful for anyone else.

Thank you @Paraphraser for that helpful gist! I have no experience with HomeAssistant, but I've managed to build, push and test docker images for the addon on all platforms, here is a simplified way to do a hack now:

  1. connect to the HA instance over ssh
  2. execute:
old_image=`docker ps -a | grep gosungrow | head -1 | awk '{print$2}'`;
new_image=`echo $old_image | awk -F/ '{print"triamazikamno/"$2}'`;
docker tag $old_image ${old_image}-backup;
docker pull $new_image;
docker tag $new_image $old_image;
  1. go to the HomeAssistant GUI, Settings, Add-ons, GoSungrow, Configuration tab and change sungrow_appkey to B0455FBE7AA0328DB57B59AA729F05D8. Restart GoSungrow when it prompts.

I can confirm, this has fixed my problem. I installed addon 'Advanced SSH & Web Terminal' Toggled "protection mode off" Started Adv SSH Web UI. Right click - Pasted the 5 lines of code. Hit ENTER. Command executed, downloaded update etc. Went to Gosungrow configuration in Add-ons and changed 'sungrow_appkey" as directed, Then I Started 'GoSunGrow"

Now Everything works !

milessw commented 6 months ago

@milessw

The problem is that, as well as changing the App Key, you also need to fake HA into using an updated image.

Please follow the steps in this comment.

Yes, thank you, I did read that, and I appreciate your effort to provide people with a possible solution. But hacking my HA installation to get a plugin to work sort of defeats the purpose of using a plugin in the first place. Also, it's not what I signed up for, nor something I would want to do.. It's unfortunate that it has headed down this track.

It would be nice to know whether the author is still maintaining this plugin. If not. It may be best for all concerned to remove it from our HA installations and explore other solutions. If the author is indeed working on a solution and planning on publishing an update or patch in the near future. It would be nice to hear from him and get an eta. Either way we really need to know what is going on.

Thanks again for your suggestion, but it's not for me.

eXplague commented 6 months ago

@milessw I understand your concern, but the whole project is developed off the back of broken security and poor api writing from Sungrow. So if they've patched it, @MickMake might not be keen to redevelop the app. If @triamazikamno or @paraphraser have a fix. Its likely fine to utilise.

quim123 commented 6 months ago

@quim123 Can you explain what to do?

Sorry for the delay. In my case, I essentially use the python script, adding the datetime library to parse the search dates, and customize the constants (credentials, ps_key, points), defining the points where the total accumulated production appears.

The result is parsed to make the necessary structure for insertion into a monitoring platform (via API).

A quick and functional solution...:)

Paraphraser commented 6 months ago

@milessw I definitely understand your point of view. At the end of the day it's always "your system, your rules."

I don't know the author so I don't know how to interpret the silence. I can't comment on whether he's working on a solution, or is occupied doing other things (eg a day job) and will get to it eventually, or has abandoned the project, or any other possibility.

I suppose what fascinates me about Open Source projects is how the community of interest pulls together to find solutions, even if those solutions are interim and slightly suboptimal. It's certainly what motivates me to try to contribute to projects.

Cheesekeeper commented 6 months ago

@triamazikamno That's seriously awesome, thank you. Mine is back up and running again. Greatly appreciated.

triamazikamno commented 6 months ago

The commands provided by @triamazikamno rely on docker ps. I think that strategy will only work if the container is running.

Hm, that is strange, it should be taken care of with the -a flag already: -a, --all Show all containers (default shows just running) I've just tested on a stopped container, it works fine. I wonder if HA completely removes container in some conditions? What does docker ps -a return for you @Paraphraser when the pluggin is stopped?

khkissel commented 6 months ago

I've made a very dirty and quick Go port of those python implementations: triamazikamno@5bb253c Feel free to build from the encryption branch of my fork. I've done very limited testing, but so far it's working fine for me.

Note: you need to change appkey in your config.json to B0455FBE7AA0328DB57B59AA729F05D8

A lot of people are following this issue in the hope than an official update will be released. Although the side-track into ModbusTCP2MQTT is fascinating (and I'll probably try it on my own WiFi-connected SG5.0RS sometime), I think there's a small risk that the original er_invalid_appkey problem is being buried.

So, just for now, I'd like to haul the discussion back to the original problem.

Earlier, @triamazikamno wrote:

I've made a very dirty and quick Go port of those python implementations: triamazikamno/GoSungrow@5bb253c Feel free to build from the encryption branch of my fork. I've done very limited testing, but so far it's working fine for me. Note: you need to change appkey in your config.json to B0455FBE7AA0328DB57B59AA729F05D8

I can confirm that this is working for me too. The earlier post was a little short on the how-to so I've updated this gist on compiling GoSungrow to explain how I applied that to my own situation.

The way I use GoSungrow is from a cron job that fires once a day to fetch "yesterday's" data which I munge into an SQLite database. So, for me, just being able to recompile with the excellent fix from @triamazikamno has gotten me going again. I was able to download all missing data since everything turned to custard.

I am not a big HomeAssistant user. I have it running but only for tinkering and I can't claim anything more than the shallowest of shallow knowledge of the whole HA ecosystem.

I do, however, have a reasonable amount of Docker knowledge so I was able to apply that to add the recompiled GoSungrow binary to the base image from MickMake that was already on my HA system, and get it to work.

For all I know a HomeAssistant guru would take one look at my approach and say, "oh, that's doing it the hard way - you just need to …". If anyone reading this is such a guru and is able to enlighten me as to the True Path, I'd really welcome that because this kind of problem (of having a fix in advance of a formal release) is bound to come up again.

In the meantime, I've added my hack to the same gist in case the same approach is useful for anyone else.

This works for me too. Great work, thanks a lot.

Paraphraser commented 6 months ago

@triamazikamno I think it's the difference between "container crashed" and "container stopped". Both show up in the GUI as "START" but you can generally tell the difference when you look at th Log tab (no log means stopped).

Anyway, container stopped:

IMG_2413 IMG_2415

Container running:

IMG_2414 IMG_2416

If a crash turns up I'll add that. I have actually had one since the "upgrade" but I didn't capture it. I assumed it was a one off but if it recurs I'll definitely grab some evidence.

triamazikamno commented 6 months ago

Thank you @Paraphraser, I was able to reproduce it now. It does indeed remove container completely when you stop the plugin. This should do it then :)

docker image list --format 'table {{.Repository}}:{{.Tag}}' | grep gosungrow | grep -vE 'triamazikamno|backup' | head -1                                                                                                                                                                                                              

I'll edit my original https://github.com/MickMake/GoSungrow/issues/101#issuecomment-1845634809

Paraphraser commented 6 months ago

@triamazikamno - I suspect that what @milessw wrote earlier represents the view of a reasonable number of GoSungrow users so I'm curious about what you see as the next logical step?

Let's suppose @MickMake applies your fixes and publishes v3.0.8. My guess is that the HA GUI will show that and invite the update. If so, the result will probably be 3.0.7-backup and your custom 3.0.7 left dangling, waiting for each user to do a manual cleanup. Do you agree or do you think the current situation might actually block HA being able to apply updates?

Now let's suppose the opposite: @MickMake abandons the project. Will you want to take over the project and maintain your fork, or do you see that as a bit too much of a commitment? I'm not wanting to apply any pressure one way or the other. I'd just like to know whether you've given this any thought?

If you did wind up taking over the project, how would you see the HA "add-on" store side of things working? I don't know enough about HA's policies to know whether it's feasible to "take over" an existing add-on, or if you'd have to invent something like "GoSungrow2"?

If you would not want to take over the project then it's probably important to know that too so that people who currently depend on GoSungrow can consider switching to ModbusTCP2MQTT.

Paraphraser commented 6 months ago

@triamazikamno - I could add that a repo I do a fair bit of work on had to be forked when the originator just disappeared. To this day we don't know what happened. It was in 2020 right as COVID was getting into full swing and the person had been travelling so it's possible he was a casualty. It's also possible the responsibility became too big and he walked away without saying a word. Whatever the explanation, it caused a fair bit of disruption and the lesson I learned is it's never to early to start thinking about succession planning.

wberbert commented 6 months ago

Worked for me thanks

https://github.com/MickMake/GoSungrow/issues/101#issuecomment-1846484313

threepoints85 commented 6 months ago

@triamazikamno Thank you very much. Its finally working again

triamazikamno commented 6 months ago

@Paraphraser - You are raising valid points. I don't use HA myself and have zero experience with it, except for installing in order to build and test the plugin images, but to me it looks like when the new version is released, it should just ask for new update, since the hack is using same version and same image name. Looks pretty harmless. Users would indeed need to clear those dangling images manually - docker rmi <IMAGE_NAME>. Regarding the maintenance, I don't think that I'm the right person unfortunately, since I'm not a HA user. I hope that @MickMake comes back! @MickMake if you just don't have time to look in to it, please let us know! I can make the hack less hacky and prepare a proper PR that you would quickly review and release new version.