MickMake / GoSungrow

GoLang implementation to access the iSolarCloud API updated by SunGrow inverters.
https://mickmake.com/
GNU General Public License v2.0
148 stars 42 forks source link

Can I use this in a HA installation without supervisor? #32

Open stalakerob opened 1 year ago

stalakerob commented 1 year ago

Hi, I'd like to use this component in my HA installation but I'm running HA as a docker installation on a raspberry Pi 4. There is no supervisor, so I cannot use add-ons. Is there a docker file for Linux (raspberry PI OS) on arm ? If not, can I create one and if so, how? Thx

scarafaggio commented 1 year ago

Hi, did you make any progress on this? I have a similar situation, HA in a docker container on another box.
I've been fiddling trying to get a Dockerfile working, but my main challenge is that I've never touched GO before, so learning that at the same time.
There are 2x Dockerfiles in the repo. One uses local files and the other remote. In the local case the docker build seems to work until the copy step can't find a dist/ directory. In the remote case wget falls over with a 404/not found error. In either case I can't find a GoSungrow executable.
Alpine doesn't have GO installed, and there is no mention that I can see elsewhere of a "build" step to generate the binary. So where does that GO build/compile step happen? Presumably I'm misunderstanding how this is supposed to work.
I haven't got as far as the ssh certs yet ... but that is looking tricky too! Anyway, if you make any progress please share ... good luck.

stalakerob commented 1 year ago

Thanks for posting your interest as well. Unfortunately no progress so far. It should not be hard to do that since a HA add-on is also a container image. Most add-ons are also available as a docker image but this one is not. I do not have enough technical skills to convert this add-on. It should be in the interest of the developer to offer that. It could then be used not only for HA installations.

Paraphraser commented 1 year ago

@scarafaggio

Alpine doesn't have GO installed, and there is no mention that I can see elsewhere of a "build" step to generate the binary. So where does that GO build/compile step happen? Presumably I'm misunderstanding how this is supposed to work.

I can help you with this bit. See this gist.

Once you have compiled GoSungrow, you can move the compiled binary into the directory alongside the Dockerfile and use the Dockerfile COPY command to copy it into the image as you build it.

There are some rules about what gets copied into the image as part of the build context. I don't have a full understanding but I interpret it to mean "minimal is best". In other words, it is better to move the compiled binary next to the Dockerfile, rather than clone the GoSungrow repo into the same directory as the Dockerfile, compile it there, and then have COPY reference ./GoSungrow/GoSungrow - if that makes sense.

I haven't got as far as the ssh certs yet ... but that is looking tricky too!

I'm very very new to Sungrow and GoSungrow so I haven't gotten my brain around very much of anything - including this bit.

What are the "ssh certs" for?

If it's just so the process inside the container can run commands outside the container using ssh then you should be able to follow the instructions here to generate any necessary keys in situ.

Assuming the process inside the container runs as root, its home directory will be /root. If that internal path is mapped to a persistent store outside the container, both /root/.ssh and /root/.GoSungrow should persist across invocations.

Alpine doesn't have …

On the topic of Alpine, I know it's lightweight and fashionable as an image base but it seems to come with a lot of gotchas. I spend a lot of time working on IOTstack and, over the past 3-odd years, Alpine has wound up being implicated in more than its fair share of issues.

At the moment I'm following a discussion on Node-RED where DNS failover doesn't work. That has been tracked to Alpine.

There's also a proposal to build a Debian version which is based on that having solved a fairly weird problem that showed up on Alpine. The proposal is getting pushback on the basis of workload. That's not an unreasonable position to take when you're the one who will have to do the work but it isn't actually an argument on the merits of the proposal.

I base my own containers on Ubuntu. Once I get them working they become "just works, rock solid, set and forget".

To put this another way, when you're building an image for your own use, the basic principle is "your container, your rules". I think "your rules" should also include "your decision" about an appropriate base OS.

Anyway, hope this helps you make progress.

scarafaggio commented 1 year ago

Thanks, your comments are very helpful. I have a bunch of containers running in a home environment. Most of these I can pull directly from dockerhub and restrict my local customisation to a few config files and docker-compose. I've done this deliberately for ease of maintenance, I can easily recreate any container or even the whole docker host. I have a few home-grown containers where the dockerfile does the work but even then I try to do everything inside the container so I'm not dependent on any other system. So in this case with GoSungrow I'd like to be able to do the compile as part of building the image. There seems to be an official image called golang which is itself based on alpine. So I'll see if I can base my container on that.

MickMake commented 1 year ago

Hi, I'd like to use this component in my HA installation but I'm running HA as a docker installation on a raspberry Pi 4. There is no supervisor, so I cannot use add-ons. Is there a docker file for Linux (raspberry PI OS) on arm ? If not, can I create one and if so, how?

Thx

Yes, it's fairly trivial to do this. I did have a Docker image file that I used to use as part of this repo. I'll have a look at updating it for general use.

MickMake commented 1 year ago

Hi, did you make any progress on this? I have a similar situation, HA in a docker container on another box.

I've been fiddling trying to get a Dockerfile working, but my main challenge is that I've never touched GO before, so learning that at the same time.

There are 2x Dockerfiles in the repo. One uses local files and the other remote. In the local case the docker build seems to work until the copy step can't find a dist/ directory. In the remote case wget falls over with a 404/not found error. In either case I can't find a GoSungrow executable.

Alpine doesn't have GO installed, and there is no mention that I can see elsewhere of a "build" step to generate the binary. So where does that GO build/compile step happen? Presumably I'm misunderstanding how this is supposed to work.

I haven't got as far as the ssh certs yet ... but that is looking tricky too!

Anyway, if you make any progress please share ... good luck.

Yeah, these are the docker files that I mentioned previously. I used them for testing, but I should update them properly.

Instead of rebuilding from scratch, I would use the prebuilt binaries that I create with every release.

That way, it's a much quicker docker build for you.

stalakerob commented 1 year ago

Anyone got a docker build running? If so, how?

Altau38650 commented 1 year ago

Hello,

I'm new to HA and also to Docker and I don't speak English very well. However, I think I understood that, MickMake, you were working on a version of your tool to be installed on HA Docker?

So I am interested in this solution. When do you think you can provide this tool?

Many thanks for your work and for your future reply.

-- Alain France Inverter : SG3-0RS

stalakerob commented 1 year ago

@Altau38650 Alternatively you can use https://github.com/mkaiser/Sungrow-SHx-Inverter-Modbus-Home-Assistant

Altau38650 commented 1 year ago

Thank you very much for the link. I have managed to get this system to work in order to track my solar production. Good luck with that.

Alain

EvoSems commented 5 months ago

ok. compiled myself on a rpi! big yay! now what to do to make a docker file?

Paraphraser commented 5 months ago

@EvoSems You don't actually have to create your own Dockerfile. You can just re-use the images that were placed on DockerHub by triamazikamno. They are just "Docker images" so it doesn't really matter all that much whether it's Home Assistant or docker-compose or a docker pull command doing the "pulling".

Here's a recipe for you to follow:

  1. Add the following service definition to your docker-compose.yml:

      gosungrow:
        container_name: gosungrow
        image: triamazikamno/armv7-addon-gosungrow:3.0.7
        # image: triamazikamno/aarch64-addon-gosungrow:3.0.7
        # image: triamazikamno/armhf-addon-gosungrow:3.0.7
        # image: triamazikamno/amd64-addon-gosungrow:3.0.7
        restart: unless-stopped
        environment:
          - TZ=${TZ:-Etc/UTC}
        volumes:
          - /etc/ssl/certs:/etc/ssl/certs:ro
          - ./volumes/gosungrow/data:/data
  2. Activate the image clause which is appropriate to your hardware.

  3. The service definition above follows IOTstack conventions for the location of persistent storage:

    • In ./volumes/gosungrow/data the leading . means "the path to the folder containing docker-compose.yml" which is usually ~/IOTstack so the expected folder structure is:

      ~/IOTstack
      ├── docker-compose.yml
      └── volumes
          └── gosungrow
              └── data
    • By default, docker-compose will create that structure on first launch with the equivalent of:

      $ sudo mkdir -p ~/IOTstack/volumes/gosungrow/data

      However, for reasons I'm about to get to, you should:

      1. Decide whether that folder structure is appropriate to your situation and adapt the left-hand-side path as necessary, and then
      2. Create the folder structure yourself before launching the container for the first time.
  4. The DockerHub images for GoSungrow assume that an options.json file will be provided externally. When you add GoSungrow to Home Assistant, this happens as a by-product of the "Configuration" tab (in Settings » Add-ons » GoSungrow). If you want to use one of these images outside of the HA environment, you need to satisfy this configuration dependency yourself. Use the following as a template:

    {
      "sungrow_host": "https://augateway.isolarcloud.com",
      "sungrow_user": "«your Sungrow user name here»",
      "sungrow_password": "«your Sungrow password here»",
      "sungrow_appkey": "B0455FBE7AA0328DB57B59AA729F05D8",
      "mqtt_host": "mosquitto",
      "mqtt_port": "1883",
      "mqtt_user": "",
      "mqtt_password": "",
      "debug": false,
      "sungrow_timeout": 60
    }
    • change the sungrow_host if your inverter logs to a different cloud server.
    • set your username and password for the Sungrow host.
    • the appkey shown here is the new one (ie correct at the moment).
    • the mqtt_host assumes the Mosquitto container is running on the same Docker instance and is reachable across the internal docker bridged network via the mosquitto hostname. If your Mosquitto instance runs somewhere else, set this to the relevant domain name or IP address.
    • if your Mosquitto instance checks credentials then set mqtt_user and the associated password to something valid, otherwise leave them blank.
  5. Once the JSON template has the correct values, save it at the path:

    ~/IOTstack/volumes/gosungrow/data/options.json

    or whatever path prefix you decided upon in step 3, remembering that the data folder is owned by root so you will need sudo to create the file (I don't think the ownership of the options.json file matters).

  6. Bring up the container. For example:

    $ cd ~/IOTstack
    $ docker-compose up -d gosungrow

What I find when I do this on a Raspberry Pi 4 (Bullseye, full 64-bit) is that:

  1. It works (as evidenced by Mosquitto receiving the telemetry and me being able to subscribe to the topics); but
  2. docker-compose moans that:

    the requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

This does not happen when I repeat the experiment on a Debian Bookworm guest running on Proxmox-VE on Intel hardware.

I think the Raspberry Pi behaviour is probably due to a small bug in the process @triamazikamno used to build the original images on DockerHub. The warnings are independent of whether I use the armv7 or aarch64 image variants, both of which are "correct" for the hardware platform. The fact that each ARM-specific image works (in the sense of running, connecting to the cloud, and publishing to MQTT topics) means the compiled code inside the image is actually correct for the hardware. In other words, if it really was AMD code, it wouldn't run on ARM.

I mention this in case you try it on ARM (which is what you seem to be doing). Although seeing the same warnings repeated multiple times on each "up" is a bit scary, I am pretty sure it is safe to ignore them. My guess is they are occurring in the HA environment too when it's an ARM platform but I don't have any means of testing that theory.

Hope this helps.

EvoSems commented 5 months ago

awesome thanks for the response! I have got some progress but I am getting an error logging in? I have checked my credentials over and over. any ideas?

$ sudo docker run -d --name gosungrow --restart unless-stopped -v /home/evo/gosungrow:/data/ -v /home/evo/gosungrow/certs:/etc/ssl/certs:ro -e TZ=Australia/Brisbane --network=host triamazikamno/armv7-addon-gosungrow:3.0.7 WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 83c3f10683d150e974a5597ceba4b5d9c4da9d13701213aacce24e8a06e7ecdc

manually added the following structure/files home/evo/gosungrow/.GoSungrow/config.json (with my details in full included) home/evo/gosungrow/options.json (details included)

error logs.

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
[02:18:39] INFO: Setting up GoSungrow config ...
[02:18:40] WARNING: MQTT user is empty. I'm guessing this is what you wanted?
[02:18:40] WARNING: MQTT password is empty. I'm guessing this is what you wanted?
[02:18:40] INFO: Writing GoSungrow config ...
New config:
Using config file '/data/.GoSungrow/config.json'
(((removed credentials from post)))
[02:18:43] INFO: Login to iSolarCloud using gateway https://augateway.isolarcloud.com ...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x4b69a8]
goroutine 1 [running]:
github.com/MickMake/GoSungrow/iSolarCloud/api.(*Web).getApi(0x5120900, {0x253e0e0, 0x5121200})
    /Users/isit/git/GoSungrow/iSolarCloud/api/web.go:279 +0xb64
github.com/MickMake/GoSungrow/iSolarCloud/api.(*Web).Get(0x5120900, {0x253e0e0, 0x5121200})
    /Users/isit/git/GoSungrow/iSolarCloud/api/web.go:74 +0x1a0
github.com/MickMake/GoSungrow/iSolarCloud/AppService/login.EndPoint.Call(...)
    /Users/isit/git/GoSungrow/iSolarCloud/AppService/login/struct.go:138
github.com/MickMake/GoSungrow/iSolarCloud/AppService/login.(*EndPoint).Login(0x5120058, 0x524d900)
    /Users/isit/git/GoSungrow/iSolarCloud/AppService/login/auth.go:99 +0x4a8
github.com/MickMake/GoSungrow/iSolarCloud.(*SunGrow).login(0x5120000)
    /Users/isit/git/GoSungrow/iSolarCloud/struct.go:312 +0x1b0
github.com/MickMake/GoSungrow/iSolarCloud.(*SunGrow).Login(0x5120000, {{0x5029a60, 0x20}, {0x502bdd0, 0x13}, {0x5015f00, 0xc}, {0x0, 0x0}, 0x0, ...})
    /Users/isit/git/GoSungrow/iSolarCloud/struct.go:330 +0x1e8
github.com/MickMake/GoSungrow/cmd.(*CmdApi).ApiLogin(0x501e0e0, 0x0)
    /Users/isit/git/GoSungrow/cmd/cmd_api.go:396 +0xf4
github.com/MickMake/GoSungrow/cmd.(*Cmds).SunGrowArgs(0x4017be0, 0x5032300, {0x402911c, 0x0, 0x0})
    /Users/isit/git/GoSungrow/cmd/cmd_api.go:291 +0x218
github.com/spf13/cobra.(*Command).execute(0x5032300, {0x4cb4010, 0x0, 0x0})
    /Users/isit/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:925 +0x688
github.com/spf13/cobra.(*Command).ExecuteC(0x4f29500)
    /Users/isit/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x418
github.com/spf13/cobra.(*Command).Execute(...)
    /Users/isit/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/MickMake/GoUnify/Unify.(*Commands).Execute(...)
    /Users/isit/go/pkg/mod/github.com/!mick!make/!go!unify@v1.0.3-0.20230904042338-0db745f1bada/Unify/struct.go:277
github.com/MickMake/GoUnify/Unify.(*Unify).Execute(0x4ca6780)
    /Users/isit/go/pkg/mod/github.com/!mick!make/!go!unify@v1.0.3-0.20230904042338-0db745f1bada/Unify/struct.go:216 +0x398
github.com/MickMake/GoSungrow/cmd.Execute(...)
    /Users/isit/git/GoSungrow/cmd/commands.go:94
main.main()
    /Users/isit/git/GoSungrow/main.go:11 +0x74
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
EvoSems commented 5 months ago

also any news on modbus support? I see it in the config fields. : )

and another thing is. I can't add my solar to HA energy panel. the sensors I need are not options in the selection panel?

ex. sensor.gosungrow_virtual_xxx_1_1_2_p24 (My Solar stats)

this is not selectable or even visible in the HA energy settings page for solar. same with grid. sensor.gosungrow_virtual_xxx_7_1_2_p8018 sensor.gosungrow_virtual_xxx_p83106

Paraphraser commented 5 months ago

I can't say for certain but perhaps try again save that, this time, leave the /etc/ssl/certs:/etc/ssl/certs:ro volume mapping exactly as written. What it's doing is mapping your host's certs folder (which is maintained by the OS vendor - eg Raspberry Pi Foundation) through to the container (with the :ro read-only suffix protecting the host's folder from interference from the container - always guard against rogue processes). I suppose you could always copy the host's /etc/ssl/certs to a folder of your choice and map that Into the container but then you would also be responsible for keeping that copy in parallel with the original. It's much simpler to let routine apt update ; apt upgrade do the work for you.

Paraphraser commented 5 months ago

also any news on modbus support? I see it in the config fields. : )

I see it too but I've no idea. I can't speak for MickMake. I just hang out here (plus a few other GitHub repos) and answer the odd question if it's within my skill set - my main motivation is that I always learn something when I attempt to answer a question.

and another thing is. I can't add my solar to HA energy panel. the sensors I need are not options in the selection panel?

ex. sensor.gosungrow_virtual_xxx_1_1_2_p24 (My Solar stats)

this is not selectable or even visible in the HA energy settings page for solar. same with grid. sensor.gosungrow_virtual_xxx_7_1_2_p8018 sensor.gosungrow_virtual_xxx_p83106

Look, I had the same problem too. Just over a year ago I replaced a SolaX with the Sungrow and all I really wanted to do with GoSungrow was continue the same data set. It took me a while to figure out which metrics in the vast quantity of what's available were the ones I needed. A side effect of that process brought the surprising realisation that much of what was useful to me didn't actually show up in any of what I'll call GoSungrow's standard outputs. Kinda weird.

Anyway, I just have a cron job which fires at 3am to invoke GoSungrow to download "yesterday's" data and munge it into a CSV which winds up in an SQLite database. Ultimately, all I really care about is, for a given billing period, what did the panels produce (both DC and AC), what was self-use, what was the grid usage, and the feed-in (for a pissy 7 cents/kWh). That's enough to calculate both "what will the next bill be?" and "what would it have been without solar?". Also, I'm never going to be more than a day from discovering if the system has gone on the blink (hasn't happened yet but...).

If I need "live", I just use the iSolarCloud app.

In short, I'm a bit of a low-rent user; definitely lacking in sophistication. 🤓

rohanrichards commented 2 months ago

@EvoSems You don't actually have to create your own Dockerfile. You can just re-use the images that were placed on DockerHub by triamazikamno. They are just "Docker images" so it doesn't really matter all that much whether it's Home Assistant or docker-compose or a docker pull command doing the "pulling". ...

Just wanted to say a huge thank you, I followed this guide and it worked perfectly first time. It sort of half works in the energy tab, the graphs look fine but the card data seems off. My recommendation is to just peak at all the sensor info and try to find the numbers that make sense. Though I never know if the energy tab wants totals or real-time data. I have just ended up making my own energy tab: image

Really happy with this result, considering I went down the modbus rabbit hole, and learned the hard way my wiNet module did not like having modbus over wifi connections and refused to reconnect 😬. I'm so glad to have found goSungrow as an alternative for people who can't get LAN access to their inverter.