GoodWe (SEMS) support

[T-wijk]

GoodWe changed to SEMS portal to provide output. I can't find the right code to connect my panels.

[DiedB]

I see and I'll be happy to implement it. Send me an email at d@iederik.nl with (temporary) login details and I'll have a look whether Homey can be coupled to the new portal.

[cesarenl]

Having the same problem, happy to help if you need any information.

[macfreek]

@DiedB I'm happy to share my login details, and send you an email telling so.

Good starting point for the implementation is https://pastebin.com/4UNEmG4v.

[DiedB]

Scraping the SEMS portal is not a favourable solution. It can break at any time (and I do not have the time to support that), and it would cost quite some time (since there is no existing JS implementation of a scraper).

I would suggest you to install a smart meter in your electricity panel to measure your solar panels, as it gives more detailed information and will keep working all of the time. An example: https://qubino.com/products/smart-meter/

[iformarco]

Ok. Maybe a good idea to remove the Goodwe-option in the Homey-app to avoid more questions?

[DiedB]

Yes, it will (unfortunately) be cleaned up in the next version.

[macfreek]

Hi @DiedB, I took a quick look at the code and the MEMS portal. What you can do is (1) log in to a page with credential and get a session id, (2) fetch a URL that outputs JSON data with power measurement data.

The word "scraper" seems to imply some sort of HTML parser. For clarity, that is not needed.

Would you accept a code contribution that adds support for the MEMS portal? I'm happy to create a pull request (although with medium priority, so expect something in a month or so), although I would appreciate some help in how the SolarPanels app works. In particular, the API between the core and the vendor-specific code: how often is it a method called, and what should be the resulting data format. For example, should it return only the last measurement result, or as much data as provided by the vendor-API (in this case, the MEMS portal)?

[DiedB]

Cool, that sounds better. Although not as robust as an API (as those will not change suddenly), it sounds good enough for now.

I will be developing a beta for 3.0 (within two weeks), which contains a few tweaks to the shared code of the app and its capabilities. I am definitely willing to accept a pull request for GoodWe after that!

Solar Panels uses cron tasks to regularly poll for data. These trigger a task that fetches the current value from the manufacturer. If a manufacturer provides high-resolution data, then the interval of the cron task is lowered to match.

[thezahir1]

Any progress in this issue?

[hspenk]

Any progress in this issue?

Would also love to have GoodWe - SEMS support back!

[DiedB]

v3.0.2 has been released, you can fork the master branch of the repository and start programming (@macfreek).

[thezahir1]

@macfreek great to learn that you are going to bring back goodwe support to the app! Any idea of ETA for this?

[thezahir1]

@macfreek or @DiedB any chance you can share an update about the return of Goodwe support? Really miss it!

[dutchdevil83]

There is a solution for domoticz can anyone make this work for homey?

https://www.domoticz.com/forum/viewtopic.php?f=61&t=27670

[meden199]

I'm using the GW2PVO script (https://github.com/markruys/gw2pvo) on a PI.

This script is uploading my Goodwe (sems portal) data to PVoutput. SolarPanels app in homey connected to my PVOutput and Problem Sovled (For me)

[ferdyh]

The GW2PVO uses the 'new' api of the semsportal, and it appears to contain all the information needed. Documentation can be found: http://euapi.sems.com.cn:82/swagger/ui/index Example from gw2pvo: https://github.com/markruys/gw2pvo/blob/master/gw2pvo/gw_api.py Hope this helps

[DiedB]

Thank you! I'll take a few hours to look into it once I have the time.

[ferdyh]

Lovely. If you need someone to test a branch, just let me know.

[macfreek]

My apologies for not checking in earlier. My solar panels are on a new home, and other jobs still have priority. I'm sure you don't care that my backyard is now a green lawn rather than a wasteland, but my wife did care ;)

I did create a script to see if I can get some output. Here are some details.

You need to following URL endpoints:

Log in

First, you need to get a authentication token for further requests, if you don't have one.

Post to https://globalapi.sems.com.cn/api/v1/Common/CrossLogin With the headers:

'Content-Type': 'application/json'
'Connect': 'keep-alive'
'User-Agent': 'PVMaster/2.1.0 (iPhone; iOS 12.0; Scale/2.00)',
'Accept-Language': 'nl-BE;q=1',
'Token': '{"client":"ios","version":"v2.1.0","language":"en"}',

And with the body: {'account': 'email@example.org', 'pwd': '123456'}

Note that this 'Token' stuff in the header with client information is required. If it is empty or left out, I did not get a result. Or an error message in Chinese. And I don't know why it is called 'Token' either. It has nothing to do with the authentication token you'll receive in the response.

Data returned looks like:

{ 
  "hasError": false, 
  "code": 0, 
  "msg": "Success", 
  "data": { 
    "uid": "c3518cbe-e6b8-4810-c15b-b1f48f32288f", 
    "timestamp": 1565735721330, 
    "token": "25b81fd55a3e45eb0434301cbfb0ea3d", 
    "client": "ios", 
    "version": "v2.1.0", 
    "language": "en" 
  }, 
  "components": { 
    "para": null, 
    "langVer": 40, 
    "timeSpan": 0, 
    "api": "http://eu.semsportal.com:82/api/Auth/GetToken" 
  }, 
  "api": "https://eu.semsportal.com/api/" 
}

From this, you need to store:

• data > uid
• data > token
• data > timestamp
• api (thus not components > api)

I have not determined how long the authentication token remains valid, but I get the impression not forever. Based on how often I need to log in on my computer, I expect 12 to 24 hours.

Get Plant ID

For the first time solar panels are added, you need to get the ID of the 'solar plant'. After this first request, you can store the plant ID.

Post to {api}PowerStationMonitor/QueryPowerStationMonitorForApp, with the api URL you got when you logged in. Likely: https://eu.semsportal.com/api/PowerStationMonitor/QueryPowerStationMonitorForApp, With the headers:

'Content-Type': 'application/json'
'Connect': 'keep-alive'
'User-Agent': 'PVMaster/2.1.0 (iPhone; iOS 12.0; Scale/2.00)',
'Accept-Language': 'nl-NL;q=1',
'Token': '{'Token': '{"client": "ios", "version": "v2.1.0", "language": "en", "timestamp": 1565735721330, "uid": "c3518cbe-e6b8-4810-c15b-b1f48f32288f", "token": "25b81fd55a3e45eb0434301cbfb0ea3d"}'}',

And with the body: {'page_size': '5', 'orderby': '', 'powerstation_status': '', 'key': '', 'page_index': '1', 'powerstation_id': '', 'powerstation_type': ''}

With the Token header updated with the information you got when logging in.

Data returned looks like:

{ 
  "hasError": false, 
  "code": 0, 
  "msg": "Success", 
  "data": [ 
    { 
      "powerstation_id": "e9b465a8-d2fb-4e41-9348-ee9f49501a03", 
      "stationname": "Watermunt", 
      "first_letter": "", 
      "adcode": "011300140010", 
      "location": "", 
      "status": -1, 
      "pac": 0.0, 
      "capacity": 4.5, 
      "eday": 13.1, 
      "emonth": 195.0, 
      "eday_income": 604.318, 
      "etotal": 2746.9, 
      "powerstation_type": "residential", 
      "pre_org_id": null, 
      "org_id": null, 
      "longitude": "4.95537042617798", 
      "latitude": "52.1518383717057", 
      "pac_kw": 2746.9, 
      "to_hour": 2.911111111111111, 
      "weather": { 
        "HeWeather6": [ 
          { 
            "basic": { 
              "cid": "NL2758333", 
              "location": "Breukelen", 
              "parent_city": "Breukelen", 
              "admin_area": "Utrecht", 
              "cnty": "Netherlands", 
              "lat": "52.17417145", 
              "lon": "5.00138998", 
              "tz": "+1.00" 
            }, 
            "update": { 
              "loc": "2019-08-12 23:57", 
              "utc": "2019-08-12 22:57" 
            }, 
            "status": "ok", 
            "now": { 
              "cloud": "100", 
              "cond_code": "300", 
              "cond_txt": "Shower Rain", 
              "fl": "14", 
              "hum": "100", 
              "pcpn": "0.3", 
              "pres": "1014", 
              "tmp": "14", 
              "vis": "16", 
              "wind_deg": "90", 
              "wind_dir": "E", 
              "wind_sc": "1", 
              "wind_spd": "4" 
            } 
          } 
        ] 
      }, 
      "currency": "EUR", 
      "yield_rate": 0.22, 
      "is_stored": false 
    } 
  ], 
  "components": { 
    "para": null, 
    "langVer": 40, 
    "timeSpan": 0, 
    "api": "http://eu.semsportal.com:82/api/PowerStationMonitor/QueryPowerStationMonitorForApp" 
  } 
}

From this, you need to store:

• data > [item 0] > powerstation_id

Optionally, you can ask the user which power station to use. Likely very few people will have more than one, but by listing the options and name, it is immediately apparent for the user that log in was successful.

Get Current Data

Post to {api}v1/PowerStation/GetMonitorDetailByPowerstationId, with the api URL you got when you logged in. Likely: https://eu.semsportal.com/api/v1/PowerStation/GetMonitorDetailByPowerstationId, With the headers:

'Content-Type': 'application/json'
'Connect': 'keep-alive'
'User-Agent': 'PVMaster/2.1.0 (iPhone; iOS 12.0; Scale/2.00)',
'Accept-Language': 'nl-NL;q=1',
'Token': '{'Token': '{"client": "ios", "version": "v2.1.0", "language": "en", "timestamp": 1565735721330, "uid": "c3518cbe-e6b8-4810-c15b-b1f48f32288f", "token": "25b81fd55a3e45eb0434301cbfb0ea3d"}'}',

And with the body: {'powerStationId': 'e9b465a8-d2fb-4e41-9348-ee9f49501a03'}

Now you get the result as shown in https://pastebin.com/JnMJzX6N.

Note that I have two inverters, the first with one circuit, the second with two circuits. I have not yet studied the output, but it would be great if I could see the power generated by the 3 individual circuits.

Note that this output was taken at night, it will most like not contain useful data. I try to run it again during daytime. The linked pastebin contains a full exchange with the API during daytime.

The total current power (all inverters) seems available in data > kpi > pac

The relevant output seems at data > inverter > [item in list], and then either d or invert_full (data seems the same), and for individual circuits, multiply voltage and current. Input (DC) voltage is at vpv1, vpv2, vpv3, vpv4 (for up to 4 trackers), and input current at ipv1, ipv2, ipv3, ipv4. Similar, output voltage and current are in vac1, vac2, vac3 and iac1, iac2, iac3 respectively. I'm not sure about the accuracy, but it could even be possible to determine how effective the inverter is :).

Get Historic Data

To get historic data, use either one of these URLs:

https://eu.semsportal.com/api/PowerStationMonitor/GetPowerStationPowerAndIncomeByDay https://eu.semsportal.com/api/PowerStationMonitor/GetPowerStationPacByDayForApp

I've seen some code use the first URL, other code the other URL. I have not found time to examine the difference.

[DiedB]

@macfreek Thanks a lot for the detailed report, very useful! I will put this on my todo list and implement it as soon as possible.

[macfreek]

@macfreek Thanks a lot for the detailed report, very useful! I will put this on my todo list and implement it as soon as possible.

Again, feel free to ask for my credentials by email. My apologies that I did not find the time to actually implement it in node.js. I'll update the linked pastebin and above comment with some more details if I have them.

[DiedB]

@macfreek You can send them to me at diederik@outlook.com. Thanks in advance!

[DiedB]

I've made some progress on this, but it is very noticeable that this is not the way GoodWe intended use of their API. Main issue I'm having is that the Athom SDK does not allow for communication between devices. Meanwhile, the GoodWe API invalidates all old tokens when a new one is requested. This results in a race condition between multiple inverters to get their API tokens.

The only way to implement this would be to allow 1 device at maximum (I cannot enforce that). Using Homey with GoodWe will supposedly also break your app, in the sense that you have to login to the app every time you use it (and even while you're using it).

[macfreek]

Diederik, awesome that you put some time into it.

Let me see if I understand correctly: if you want each inverter represented as an individual device in Homey, each uses a new API key, and because GoodWe only allows only one active API key, this means each Homey Device needs to log in. I assume you can get this to work by storing username and password, but this is indeed not ideal.

I presume that having all inverters in a plant represented as a single device in Homey has other disadvantages (e.g. your app currently only has a single power output per device, and making this more flexible is a lot of work).

There are apps that create multiple devices in Homey for a single physical device. One that comes to mind is the Qubino Flush 2 relay (ZMNHBD1) which is represented in Homey as 3 separate devices (one for the physical device, and 2 for the 2 switches). I don't know if and how these Homey device instances communicate with each other (perhaps they don't).

[DiedB]

Saving the username and password is not the problem (it is required for this to work). The problem is that individual inverters in Homey cannot share API keys with each other (there is no central storage API, only a device-specific one as far as I know). I would like to store a central API key which can be used by multiple GoodWe devices.

In the current situation, each of the inverters will take turns in requesting a new API key on every attempt to fetch data. Because this happens on a fixed schedule (cron, on the minute), one of the inverters will not receive any data.

Representing a plant as a single device would be possible but that means that all inverters that it contains will be summed to one energy and power value. This still does not allow for the possibility to add multiple GoodWe plants that are on the same account, but that is less of a problem I presume.

I don't think the Qubino devices communicate with each other, they are just being instantiated with the same parameters.

[thezahir1]

Any news on Goodwe converter support being introduced back in this app? Very curious to hear! @DiedB and @macfreek

[o0donder0o]

Any news on Goodwe converter support being introduced back in this app? Very curious to hear! @DiedB and @macfreek

+1

[hspenk]

Any news on Goodwe converter support being introduced back in this app? Very curious to hear! @DiedB and @macfreek

+1

Also very much looking forward to this! :)

[joostvkempen]

Saving the username and password is not the problem (it is required for this to work). The problem is that individual inverters in Homey cannot share API keys with each other (there is no central storage API, only a device-specific one as far as I know). I would like to store a central API key which can be used by multiple GoodWe devices.

In the current situation, each of the inverters will take turns in requesting a new API key on every attempt to fetch data. Because this happens on a fixed schedule (cron, on the minute), one of the inverters will not receive any data.

Representing a plant as a single device would be possible but that means that all inverters that it contains will be summed to one energy and power value. This still does not allow for the possibility to add multiple GoodWe plants that are on the same account, but that is less of a problem I presume.

I don't think the Qubino devices communicate with each other, they are just being instantiated with the same parameters.

Is it correct that this problem only occurs when having multiple inverters in one house (with one Homey)? I only have 1 inverter for my house? I'm not sure what's common, for a regular dutch townhouse I think it will be just one?

[DiedB]

That is correct, and therefore this will work for 95% of users. Maybe I should just release it with that side note, but I would prefer to find a way to at least enforce that limit. It would probably mean hacking around the Athom SDK which I don't really like.

I'm just a litlle weary on releasing stuff that is not completly right/dependable. When something's not working (well), there's loads of people complaining that it doesn't work (this app has >1500 installs). That has been the case with lots of these Chinese inverter brands that have limited or no support on their APIs. On the contrary, I have had zero issues with American/European brands like SolarEdge, Enphase or Fronius.

[joostvkempen]

I understand completely, that's how they handle everything from cars to sensors ;-) They will learn over time, I hope...

For me it would be a big + if you would release it. At the moment I'm looking into removing the wifi-module from Goodwe and replacing it with a self-build-datalogger in combination with something like this https://www.ztatz.nl/. That's a lot more work ;-)

If you need any testing, I would be happy to

[DiedB]

I think I'll just release it with a big side note in the install screen. We'll see if this starts becoming a problem in the real world. Give me a week to get it released.

[DiedB]

https://homey.app/a/it.diederik.solar/test/

Please let it run for a few weeks and update me with your experiences.

[joostvkempen]

On first try there is an error 'authorization expired', after that I keep getting 'could not find pairsesson'

[DiedB]

Are you using another app or script to read out your GoodWe inverter as well? That will interfere because GoodWe only allows one API token to be valid at the same time.

It seems to work well with the credentials I received from @macfreek.

[joostvkempen]

Not at the moment, but I've been recently (<24hrs) logged in to the SEMS portal, I suppose that session hasn't expired yet? Your app should overwrite that session right?

Maybe the inverter uses the same API to upload the data to SEMS? Only this doesn't work with credentials, the inverter had a fixed ID. In the portal you add the ID as yours to your profile.

What is the easiest way to reset PairSession on Homey? So I can try some more?

UPDATE - logged in to portal and did a forced 'logout', then tried again on Homey -> it works!

I'll try it for a week and post again

[thezahir1]

Hi @DiedB , I’ve installed 4.0.2 successfully, then also succeeded in addin my (2) Goodwe inverters (I had to enter my SEMS credentials). But then I got a message when I click on the converter’s tile saying something like ‘only absolute URLs are supported’. What did I do wrong here?

[DiedB]

As indicated in this thread, what you are trying to do is not going to work unless you have separate SEMS accounts for both inverters.

[thezahir1]

Are you really sure? Because, I first added only 1 of the 2 inverters, and I also got the same error then (so before installing the 2nd converter. So I doubt if this error relates to having 2 inverters in my SEMS account.

[DiedB]

Hmm, might be another problem then. But I'm pretty sure you will only be able to use 1 inverter after we have this one solved. I will get back to you tomorrow or Friday, once I have had time to look into the issue.

[thezahir1]

Cool. Thanks @DiedB ! Looking forward to connect my panels to Homey again, after 1,5 year!

[thezahir1]

hey @DiedB despite the errors I mentioned yesterday seeing within your app, the app was still able to fetch the generated solar power from SEMS (from both my inverters, within the same SEMS account). So my 'daily solar revenue' flow is running again! Just that you know. Me happy! :)

[joostvkempen]

As promised my findings after I’v tested 4.0.2 for 10 days now. At first attempt got same errors (see above), but with second attempt it worked instantly without errors or warnings. For me it helped to login to the SEMS portal and do a forced logout (just click on ‘logout’) to make sure there were no concurrent sessions.

Since then it worked perfect, except for some time a few days ago, when the SEMS-portal itself experienced login-issues. The tile gave a warning and did not show any readings. The app/tile recovered itself after a while.

Measurements from insights are in line with the figures in the SEMS portal, so that’s ok. I’ve made some flows based on the readings (some lights on/off and notice of daily revenue).

I only have 1 converter, so not been able to test the situation with multiple converters in 1 account.

Except for the issues with the initial pairing, I think it is stable enough for release ?

Keep up the good work @DiedB

[DiedB]

I think I might have fixed the pairing bug. Please test v4.0.3 using https://homey.app/nl-nl/app/it.diederik.solar/Zonnepanelen/test/

[DiedB]

Whoops, something went wrong in version control and new bugs were introduced. Please install v4.0.4 via https://homey.app/a/it.diederik.solar/test/

[joostvkempen]

Works right away for me after updating. Did not even have to login. Tnx @DiedB

[Berrie1]

Just installed it. When adding the device and proving the Goodwe credentials, I did get the message "Could not find that PairSession". After logging off from the SEMS Portal app, the device could be added successfully.

[hspenk]

Installed it a few days ago, works like a charm! So nice to finally being able to read out my solarpanel stats. :)

[DiedB]

Support has been added in v4.0.5.