MickMake / GoSungrow

GoLang implementation to access the iSolarCloud API updated by SunGrow inverters.
https://mickmake.com/
GNU General Public License v2.0
179 stars 46 forks source link

No GoSungrow command #117

Open TrueBlueRoo opened 4 months ago

TrueBlueRoo commented 4 months ago

There is no GoSungrow command available on HA Supervisor. Couldn't locate it through a find as well.

Thanks

Paraphraser commented 4 months ago

Hi,

I know it’s not immediately obvious but there are a few thing you need to do before you can get this running on Home Assistant:

  1. Follow the instructions at MickMake/HomeAssistantAddons. That gets you as far as having the GoSungrow add-on installed. But (and this is a key point), it will not work until you’ve done the next two steps.

  2. If you have not already done so, install two other add-ons:

    • “Mosquitto broker”. The default configuration (which implies your Home Assistant username and password) is sufficient. If you have an existing MQTT broker, you can use that instead.
    • “Advanced SSH and Web Terminal”. It must be advanced. Nothing else will do. The configuration instructions for this are part of step 3 below.
  3. Follow the steps in Part 2 of this Gist.

Why all the convolution? Well, I’m not the author of this site so I don’t really know but, at a guess:

In essence, the “hack” works like this:

There are still some wrinkles involving AppKeys and picking the correct URL for where your inverter is logging its data but, once that’s all sorted out, it (mostly) starts working. That’s all covered in the last part of the Gist.

I say “mostly” because I get the sense that people who have multiple Sungrow inverters have more trouble - possibly to the extent that they either get it working and don’t say how, or never get it working and give up without reporting that they’ve given up.

Hope this helps.

TrueBlueRoo commented 4 months ago

Thanks mate. Highly appreciated.

It got the GoSungrow running now. I run it in a VM on Windows Server. That was excately that kind of tutorial what I couldn't find.

The next step I am stuck is the HA's Energy Dashboard. How do I get those sensors running like sensor.gosungrow_virtual_XXXXXXXXXXXXXX_grid_to_load_energy ?

Any help would be appreciated.

Cheers :)

Paraphraser commented 4 months ago

Sorry. Can't help you with that bit. You'll have to hope someone else with more experience chimes in to answer that bit.

In my own case, I just use the compiled binary running on a Mac (nothing to do with Home Assistant). Once a day a cron job fires to invoke GoSungrow to download "yesterday's" data at 5-minute resolution for the 12 metrics I need. I'm essentially continuing a time-series for an older SolaX inverter. I load the results of the GoSungrow fetch into an SQLite database. That's all I need and as far as I go. If I want to know what the Sungrow inverter is up to "now", I just use the iSolarCloud app.

madisorg commented 2 months ago

Hi. Im quite fresh with HA but getting Sungrow inverter data into my HA is one of the main reasons I hopped onboard. I am running HAOS and using the web interface to access it. I managed to install the superviser as well as all the other relevant add-ins. Finally, in the superviser log it says I am logged in but before the "succesful login" message there is an error message: ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /services/mqtt. Once i hit Start, nothing really happens...even after all the refreshing and restarting of HA and clearing cache, etc. Sorry again for the rookie question but thanks in advance.

Paraphraser commented 2 months ago

@madisorg

I'm just a garden-variety user with no special knowledge to please take anything I say with a few grains of salt.

First, I'm not sure how your question relates to the "No GoSungrow command" which is the topic of this issue so maybe it would have been better to have started a separate issue. I'm not saying you should go back and do that now. It's just something to keep in mind for next time.

Second, error messages generally occur in a context with other messages and the context is often really important. I've never seen anything with [supervisor.api.middleware.security] in it so I have no idea which "log" you are talking about. That makes it difficult to guess whether the Invalid token for access /services/mqtt message is the MQTT service trying to do something that HA doesn't like, or GoSungrow trying to communicate with the MQTT broker and the broker telling it to go away.

Third, and this is a wild wild guess, I think you saying "I managed to install the superviser" might represent what I'll loosely call "old thinking".

I don't really understand Home Assistant. I have it running but only so I can run GoSungrow and, in turn, only so I can try to answer questions.

I actually run GoSungrow from the command line to fetch a limited list of "yesterday's" metrics. That has nothing to do with HA or what I'm writing here.

Also, since 2019 I've been involved in the IOTstack project.

Originally, IOTstack had two options for Home Assistant. You could install:

The implication was (and is) that the container did not have the supervisor. The container was (and is) just that. A service definition in your docker-compose.yml.

The supervised version was (past tense) a separate installation that ran alongside your IOTstack services on the same machine. The IOTstack menu just happened to provide a hook for installing it.

Home Assistant has been on a long and winding track which, given how totally uninterested I am in the whole topic, sometimes does my head in. I've never really understood the terminology ("Hass.io" or "Core" or "Supervised" or blah this, blah that) which is why you shouldn't assume I actually know what I'm talking about.

Along that winding track, the "supervised" installation provided in IOTstack broke, was fixed by moving to a different (third party) installation script, then broke again, permanently.

Most of the above is summed-up here

That's around the time that HA seemed to become an "appliance". That's in the sense that you use their operating system image right from the get-go. You don't install Debian or Raspberry Pi OS or whatever and then add Home Assistant on top. You start from their image and the result is a running system. Full stop. End of discussion. They call it "Home Assistant OS".

So that's what I mean by "old thinking". I suspect you saying you "installed the superviser" might mean you're following instructions that were written (or YouTubed) in "pre-appliance" days.

I hope that makes sense.

In my case, I'm running Proxmox-VE on a 2015-era Intel MacMini. One of the guest systems is Home Assistant and I think I just followed the instructions here.

As well as GoSungrow (plus following gist part 2) my only add-ons are the "Advanced SSH & Web Terminal" and Mosquitto broker.

The only "maintenance" I have ever done is to connect to HA using either its web GUI or an iOS app and let it or the add-ons self-update.

Despite the negative vibes you might get from my general "don't care" attitude to Home Assistant, my actual experience with HA, as an appliance running on Proxmox, is that it just works. I was even pleasantly surprised, recently, when I was experimenting with adding the esphome container to IOTstack. I had provisioned an ESP32 dev board and, bingo, it just appeared like magic in Home Assistant.

If I wanted to install HA on a Raspberry Pi, I'd know I'd be dedicating the whole Pi to HA. I'd follow the instructions here.

Ditto for other hardware. Your choices boil down to:

  1. an image built for your hardware (so your hardware then becomes dedicated to HA); or
  2. a guest in a virtualisation environment like Proxmox-VE.

Hope this helps.

devonuto commented 3 weeks ago

I followed this, but I've still got no idea how or where anything is.

I still get zsh: command not found: GoSunGrow

After setting up these images:

➜  ~ docker images | grep -e REPOSITORY -e gosungrow
REPOSITORY                                            TAG            IMAGE ID       CREATED         SIZE
e4cdf632/amd64-addon-gosungrow                        3.0.7-backup   bf536b9a47c9   2 hours ago     118MB
e4cdf632/amd64-addon-gosungrow                        3.0.7          f2cbc9418287   10 months ago   161MB
triamazikamno/amd64-addon-gosungrow                   3.0.7          f2cbc9418287   10 months ago   161MB

How do I get my keys, so I can modify the yaml?

Paraphraser commented 3 weeks ago

Perhaps explain what you are trying to do because I'm confused.

It looks like you are trying to follow Gist Part 2 but the mention of zsh makes no sense in that context.

See, if you installed the "Advanced SSH and Web Terminal" add-on in Home Assistant then, irrespective of whether you accessed the terminal from the Home Assistant GUI or via SSH, you'd be running the bash shell (at least at the start).

There's no reason I can think of to switch shells plus, because of the way Advanced SSH and Web Terminal works, such a change wouldn't persist across a reboot anyway.

On the other hand, if you're trying to follow Gist Part 4 then you don't need to be following the steps that result in having three entries for GoSungrow in the docker images list.

That "three image dance" using docker tag commands is only needed to fool Home Assistant into loading the patched image. Then, if you are trying to run GoSungrow as an add-on in Home Assistant, the "where" of the configuration is the same as it is for all add-ons:

Settings » Add-ons » GoSungrow » Configuration tab

Behind the scenes that configuration tab is YAML but you never really see it unless you choose the option in Home Assistant to let you edit the YAML.

If you're trying to run the patched Docker image outside of Home Assistant then Part 4 of the gist tells you how and where to set up the configuration. If the process sounds convoluted, that's because it is convoluted. It's a by-product of the fact that the image expects to be run by Home Assistant. When you run the container yourself via docker run or docker compose you're essentially faking what Home Assistant does when it passes the contents of an add-on's configuration tab to the container at launch time. However, you need to set up a JSON configuration, not YAML.

Unless you have a good reason for wanting to run GoSungrow as a Docker container (and running it from Home Assistant is a good reason) then it is actually much easier to follow either Part 2 or Part 3 of the gist. Part 2 tells you how to compile it yourself; part 3 how to avoid compiling it yourself by downloading a precompiled binary. Either way, you get something that (a) doesn't involve a Docker container and (b) will run from the command line (even with zsh as your shell). Then you're able to follow the instructions on this repo, starting with:

$ GoSungrow config write --user=USERNAME --password=PASSWORD

which creates the ~/.GoSungrow directory and initialises config.json inside that directory. You can either use a text editor to edit config.json or you can set the app-key with:

$ GoSungrow config write --appkey=APPKEY

where your choices for APPKEY are listed here. If you are not using the Australian server, you may also need to set:

$ GoSungrow config write --host=HOSTURL

where your choices for HOSTURL are here.

Once all that is in place, you can try:

$ GoSungrow api login
$ GoSungrow show ps list

If you get sensible answers to both commands, you'll know it's worked.

I hope this goes some way to answering your question.

devonuto commented 3 weeks ago

I followed the gist, but when I try and run any gosungrow commands from the terminal, I just get an error that it's an unknown command.

I.e. there is no bin folder I can see in the home assistant directories.

So I can't run these commands

% ./bin/GoSungrow show ps list

Paraphraser commented 3 weeks ago

I'm going to make a guess and I apologise in advance if I get this wrong and wind up over-explaining things you already know. I'm also going to simplify a bit to try to avoid confusing the issue.

GoSungrow can be run in three ways:

  1. From Home Assistant as an add-on;
  2. From a Docker container; or
  3. From the command line.

The first two are the same thing. That's because what Home Assistant calls an "add-on" is a Docker container.

If you run GoSungrow as an add-on in Home Assistant, you configure it from the "configuration" tab and Home Assistant takes care of the details of starting the add-on and passing it the configuration.

That's all covered in Part 2 of the gist

If you run the container yourself using docker run or docker compose you have to mimic what Home Assistant does in order to set up and pass a configuration to the container.

That's explained in Part 4 of the gist

But, whether it's a Home Assistant add-on or you running the container yourself, the command that gets run inside the add-on/container is:

$ GoSungrow mqtt run

In mqtt mode, GoSungrow expects its configuration to include the host, port and login credentials of an MQTT broker. It then polls iSolarCloud every few minutes, re-packages your solar system metrics into MQTT payloads, and publishes them to your broker.

I'm going to repeat the substance of the last two paragraphs because it's a critical point. If you run GoSungrow as either a Home Assistant add-on or via docker run then GoSungrow runs in "mqtt mode". That's it. You don't get a choice. Full stop.

However, if you want to be able to run the entire suite of GoSungrow commands, you don't use the add-on/container, you install the binary directly. That's what Parts 1 and 3 of the gist are about (Part 1 if you want to compile it yourself; Part 3 if you don't).

Here's a demo of Part 3. I'll start by showing that GoSungrow isn't installed:

$ GoSungrow
-bash: GoSungrow: command not found

Now I'll do the steps in Part 3 of the gist. I'm doing this on a Debian system running on AMD64 architecture where GoSungrow has never been installed. I've already gone to this link and identified the URL of the package I want to download:

$ mkdir ~/GoSungrow-patched
$ cd ~/GoSungrow-patched
$ wget https://github.com/triamazikamno/GoSungrow/releases/download/v3.0.7/GoSungrow-linux_amd64.tar.gz >/dev/null 2>&1
$ tar -xzf *.tar.gz
$ file GoSungrow
GoSungrow: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=_MlzESGdVqlyHbobKIkK/je0ARJTKLYbWhyKeNzKL/Wh8Un69w8x1jnA8wIzXP/r0LeT42h1mJ88N1VWwaL, stripped

So now I have the GoSungrow binary in the local directory where I can run it using ./GoSungrow syntax. I've edited the following to protect my login credentials and other sensitive information but, other than that, this is exactly what I did.

First, I configure GoSungrow:

$ ./GoSungrow config write --user=aabbccddee --password=secrets --appkey=B0455FBE7AA0328DB57B59AA729F05D8
Using config file '/home/moi/.GoSungrow/config.json'
New config:
+-------------------+------------+---------------------------+--------------------------------+------------------------------------+
|       FLAG        | SHORT FLAG |        ENVIRONMENT        |          DESCRIPTION           |        VALUE (* = DEFAULT)         |
+-------------------+------------+---------------------------+--------------------------------+------------------------------------+
| --config          |            | GOSUNGROW_CONFIG          | GoSungrow: config file.        | /home/moi/.GoSungrow/config.json |
| --debug           |            | GOSUNGROW_DEBUG           | GoSungrow: Debug mode.         | false *                            |
| --quiet           |            | GOSUNGROW_QUIET           | GoSungrow: Silence all         | false *                            |
|                   |            |                           | messages.                      |                                    |
| --timeout         |            | GOSUNGROW_TIMEOUT         | Web timeout.                   | 30s *                              |
| --user            | -u         | GOSUNGROW_USER            | SunGrow: api username.         | aabbccddee                         |
| --password        | -p         | GOSUNGROW_PASSWORD        | SunGrow: api password.         | secrets                 |
| --appkey          |            | GOSUNGROW_APPKEY          | SunGrow: api application key.  | B0455FBE7AA0328DB57B59AA729F05D8   |
| --host            |            | GOSUNGROW_HOST            | SunGrow: Provider API URL.     | https://augateway.isolarcloud.com  |
|                   |            |                           |                                | *                                  |
| --token-expiry    |            | GOSUNGROW_TOKEN_EXPIRY    | SunGrow: last login.           |  *                                 |
| --save            | -s         | GOSUNGROW_SAVE            | Save output as a file.         | false *                            |
| --dir             |            | GOSUNGROW_DIR             | Save output base directory.    |  *                                 |
| --mqtt-user       |            | GOSUNGROW_MQTT_USER       | HASSIO: mqtt username.         |  *                                 |
| --mqtt-password   |            | GOSUNGROW_MQTT_PASSWORD   | HASSIO: mqtt password.         |  *                                 |
| --mqtt-host       |            | GOSUNGROW_MQTT_HOST       | HASSIO: mqtt host.             |  *                                 |
| --mqtt-port       |            | GOSUNGROW_MQTT_PORT       | HASSIO: mqtt port.             |  *                                 |
| --modbus-user     |            | GOSUNGROW_MODBUS_USER     | Modbus username.               |  *                                 |
| --modbus-password |            | GOSUNGROW_MODBUS_PASSWORD | Modbus password.               |  *                                 |
| --modbus-host     |            | GOSUNGROW_MODBUS_HOST     | Modbus host.                   |  *                                 |
| --modbus-port     |            | GOSUNGROW_MODBUS_PORT     | Modbus port.                   | 502 *                              |
+-------------------+------------+---------------------------+--------------------------------+------------------------------------+

Now I tell it to login. This is the "acid test".

$ ./GoSungrow api login
Email:  someone@domain.com
Create Date:    Thu Feb 09 13:14:55 CST 2023
Login Last Date:    2024-10-10 05:39:19
Login Last IP:  
Login State:    1
User Account:   aabbccddee
User Id:    666666
User Name:  Moi
Is Online:  false
Token:  666666_8b2188e9e4e345428dd1997ea0b07f9f
Token File: /home/moi/.GoSungrow/AppService_login.json

Successful login!

Tip:

Once I get a successful login, I can start fetching stuff:

$ ./GoSungrow show ps list
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Ps Key           ┃ Ps Id   ┃ Device Type ┃ Device Code ┃ Channel Id ┃ Serial #    ┃ Factory Name ┃ Device Model   ┃
┣━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┫
┃ 9999999_1_1_1    │ 9999999 │ 1           │ 1           │ 1          │ B2222334445 │ SUNGROW      │ SG5.0RS        ┃
┃ 9999999_7_1_1    │ 9999999 │ 7           │ 1           │ 1          │ B2222334445 │ SUNGROW      │ SG Smart Meter ┃
┃ 9999999_22_247_1 │ 9999999 │ 22          │ 247         │ 1          │ B2222334445 │ SUNGROW      │ WiNet-S        ┃
┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛

$ 

Now, if you want some ideas on how you might make use of being able to run GoSungrow from the command line, see Paraphraser/gosungrow-fetch.

devonuto commented 3 weeks ago

I've managed to get the home assistant etc to connect to the Sungrow API, the only thing I haven't been able to do is run any of the GoSungrow commands locally to get my "ps_key" so I can modify the yaml templates provided. I also don't know how to use these templates as I'm new to HomeAssistant.

Sounds like I need to run it from inside the docker container in home assistant via SSH?

I did notice however a lot of the entities that are referenced in the (yaml files)[https://github.com/MickMake/HomeAssistantAddons/tree/main/GoSungrow/docs/lovelace] do not exist in my mqtt entities to use. Quite a lot are not there TBH / named differently.

devonuto commented 3 weeks ago

Running inside the docker (now this doesn't make sense):

image

Paraphraser commented 3 weeks ago

OK. Full disclosure (repeating something I have said many times in issues on this repo), I don't actually use Home Assistant for anything productive. I only have the GoSungrow add-on running so I can try to help out the community by answering as many questions as I can while waiting for this repo's author to come back to us - he's the true guru. I haven't gone any further with the add-on than getting it to pull data from iSolarCloud and forward that via MQTT. I don't do anything with the data and I don't produce any graphs from it. I am just not that interested in "live" data. The other repo I pointed you at ("gosungrow-fetch") is how I actually use GoSungrow - as a compiled standalone binary running once a day from a cron job pulling down a small selection of "yesterday's" metrics at 5-minute resolution, which I load into an SQLite database. I look at it once a day but only to confirm the inverter hasn't broken in the last 24 hours.

So, "YAML templates"? No idea. Sorry. Someone else with have to help with that.

Where I have a much better understanding of things is on the Docker side.

In my earlier response I said I was simplifying things a bit to avoid confusing the issue.

Using docker exec to run GoSungrow inside the container is "confusing the issue". A lot.

Sure, the container is running and you can use either docker exec or, better, open a shell into the container by running:

docker exec -it «containerID» bash

That gets you a bash shell inside the container and, there, GoSungrow (the binary executable) is in your search PATH so you can just run GoSungrow.

BUT. GoSungrow needs a configuration file. So you have to run:

GoSungrow config write --user=USERNAME --password=PASSWORD --appkey=B0455FBE7AA0328DB57B59AA729F05D8

A screenshot as proof that it works:

container

Because you're running as root, the configuration is initialised in /root/.GoSungrow. Unless you take explicit action to do so, that path isn't persisted so everything will evaporate whenever the add-on is restarted. But it does work.

Although Docker images that are built to run as add-ons for Home Assistant, and Docker images that are built for other purposes are all "Docker images", those built to run as add-ons have a different internal construction, and that mainly centres on how the HA configuration is transported into the container when it launches.

If you want the gory details, try running this command:

docker exec «containerID» cat /usr/local/bin/run.sh

That's the start-up script and the end result is /data/.GoSungrow/config.json (which is a path inside the container).

If you want to make use of that configuration file, you can run the commands like this (and I'm assuming you've opened a shell into the container):

GoSungrow show ps list --config /data/.GoSungrow/config.json

but remember that that config file is (a) initialised from data provided by Home Assistant when it launches the container, (b) isn't bi-directional (ie if you change config.json, the changes don't propagate back to HA), (c) changes have the potential to interfere with the mqtt run command which is going on in parallel, and (d) changes won't persist across container re-creations.

If you go back and study gist Part 4 I think you will realise that it is doing exactly what you are trying to do, only better. It explains how to create a configuration file OUTSIDE the container which is transported INTO the container when you launch it, and which sets up a bind mount so everything persists. Thus, if you really want to run GoSungrow commands inside a container environment, that's the better way to do it. But, honestly, it's a heckofalot simpler to just install the binary in your PATH and get on with it.

devonuto commented 3 weeks ago

Same thing whether I use bash or shell inside the docker container, no difference. My screenshot shows I had run that, it output the values showing that username, password and apikey were populated. But it still won't let me log in, no matter how many times I re-enter those values.

I've given up on it for now. Going to use https://github.com/mkaiser/Sungrow-SHx-Inverter-Modbus-Home-Assistant instead, looks a lot more straightforward.

Paraphraser commented 3 weeks ago

Well, as you can see from the screen shot I included, it is definitely possible to login from inside the container. BUT - and this is a fairly critical point - you have to be running the patched container. Maybe go into the HA GUI, Settings. add ons, GoSungrow, Log tab and see whether it's working. The pattern of something working is repeats of the gobbledegook you see in the screen shot below (I have no idea what it means):

IMG_2802

If HA thinks the add-on is running OK then it follows the problem is in how you're initialising when you open a shell into the container. If HA is having trouble then it follows that either you're actually running the old (broken) container or the configuration parameters you are using are wrong. Username. Password. Appkey. That's the bare minimum.

Paraphraser commented 3 weeks ago

Plus, having departed from my usual strategy of showing examples as inline code rather than screenshots, I realised I managed to expose my password so I've just had to race around and change it. Lesson learned!