405 Client Error - Githubissues

gconceicao commented 10 months ago

When I try to Login the error below is happing.

requests.exceptions.HTTPError: 405 Client Error: Not Allowed for url: https://server-api.growatt.com/newTwoLoginAPI.do

Someone with the same error?

indykoning commented 10 months ago

This may be because of previous requests getting your user agent blocked. You can use another user agent or set it to generate random user agents https://github.com/indykoning/PyPi_GrowattServer#initialisation

gconceicao commented 10 months ago

It is already in Randon user ID.

self.api = growattServer.GrowattApi(add_random_user_id = True, agent_identifier =(GROWATTSERVER.GrowattApi.agent_identifier))

Dalvik/2.1.0 (Linux; U; Android 12; https://github.com/indykoning/PyPi_GrowattServer) - 21545

hossein4214 commented 10 months ago

same error , random User-Agent doesnt work for me , any other solution?

Derko01 commented 10 months ago

See https://github.com/indykoning/PyPi_GrowattServer/issues/73 This worked for me.

gconceicao commented 10 months ago

Independent of the user agent, the error persist. @Derko01 Is it work now for your?

Derko01 commented 10 months ago

version: growattServer (1.4.0) __init.py__: server_url = 'https://server.growatt.com/' my own Python-script: api = growattServer.GrowattApi(False, "whatever")

And the above combination works for me.

gconceicao commented 10 months ago

That is it.

The problem is that the Server was in server-api and not just server.

Thank you @Derko01

hossein4214 commented 10 months ago

version: growattServer (1.4.0) __init.py__: server_url = 'https://server.growatt.com/' my own Python-script: api = growattServer.GrowattApi(False, "whatever")

And the above combination works for me.

That's Worked , thank you ^_^

fridayowl commented 10 months ago

@indykoning I hope this message finds you well. I am writing to report an issue I encountered while attempting to access the Growatt API via the following server URL: https://server-api.growatt.com/

Issue Details:

Server URL: https://server-api.growatt.com/ Error Encountered: ExceptionGrowatt.py exception at login 405 Client Error: Not Allowed for url: https://server-api.growatt.com/newTwoLoginAPI.do Scenario:

When making a request to the endpoint "/newTwoLoginAPI.do" using the HTTP POST method, I am consistently met with a 405 error, indicating that the request is not allowed for the specified URL. This occurs even when I make the request without including any User-Agent in the header.

Further Observation:

In an attempt to address this, I added a random User-Agent to the request header. However, I encountered the same error message, indicating that the problem persisted.

Alternative API URL:

I also tried changing the API URL to https://server.growatt.com/. Unfortunately, this led to a different error. Specifically, upon attempting to log in, I received the following error: "No connection adapters were found for 'https://server.growatt.comnewtwologinapi.do/'."

fridayowl commented 10 months ago

@indykoning i tried using an npm library https://www.npmjs.com/package/growatt. -> This one was working for me .

Sjord commented 10 months ago

I also tried changing the API URL to https://server.growatt.com/. Unfortunately, this led to a different error. Specifically, upon attempting to log in, I received the following error: "No connection adapters were found for 'https://server.growatt.comnewtwologinapi.do/'."

This could be a result of omitting the final slash when configuring https://server.growatt.com/. I.e. you configured https://server.growatt.com instead of https://server.growatt.com/.

fridayowl commented 10 months ago

@Sjord import requests import re

def authenticate (username , password ): url = "https://server.growatt.com/login" headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36', 'Connection': 'keep-alive' } data = { 'account': username, 'password': password, 'validateCode': '' }

response = requests.post(url, headers=headers, data=data)
print(response.text)
cookies = response.headers.get('set-cookie')
print(cookies)
jsessionid_match = re.search(r'JSESSIONID=([^;]+)', cookies)
print(jsessionid_match)
if jsessionid_match:
        jsessionid = jsessionid_match.group(1)
        referer = f"https://server.growatt.com/index;jsessionid={jsessionid}"
print(referer)
return cookies , referer

def generation_get_data(asset):

auth = authenticate (asset["username"],asset["password"])
plantid= asset["plantid"]
url = f"https://server.growatt.com/panel/getDevicesByPlantList?plantId={plantid}&currPage=1"

payload = {}
headers = {
  'Cookie':auth[0],
  'Referer': auth[1]
}

response = requests.request("POST", url, headers=headers, data=payload)
return response.text

This works for me .

mvdh02 commented 9 months ago

What do I have to change for example simple.py to get this working again? For a few days ago everything worked fine. But with a server update by Growatt this error occurred to me.

muppet3000 commented 9 months ago

Yeah, if anyone works out what change needs to be made to make things work again I'd love to see it. I've noticed (from sniffing the traffic on my android phone) that it's making an additional login to oss.growatt.com now as well, but I can't see that making a difference. I can't work out how to make this library work again.

ryandrake182 commented 9 months ago

+1 for the same issue.

Things I've tried:

modify server URL: server-api.growatt.com/ (405 Client Error Not Allowed for URL)
modify server URL: server.growatt.com/ - (403 Client Error: Forbidden for URL)
secure/unsecure connection: http & https for URL (no change, same error as above)
different user-agents: (iPhone, Chrome for Windows, Safari for MacOS) (no change same error as above)
random user agent using growattServer.GrowattApi(True) (no change same error as above).
Creating a new growatt user account on my main account and using those credentials instead

Doing some quick google research it seems other libraries for pulling Growatt data are also having the same issue

Hopefully this helps someone with a bit more experience with python to fix the issue

indykoning commented 9 months ago

Yeah i'm getting errors trying to get my data from Growatt as well. At the same time i cannot use the shinephone app either to log in. It seems Growatt have tried to add certain security measures like bot detection which are backfiring. You cannot do bot detection on an API endpoint since the only things using it are going to be bots.

I do hope they fix it soon, i'm afraid we're out of user-friendly ways to get Growatt data otherwise...

This is great information though thank you!

Things I've tried:

modify server URL: server-api.growatt.com/ (405 Client Error Not Allowed for URL)

modify server URL: server.growatt.com/ - (403 Client Error: Forbidden for URL)

secure/unsecure connection: http & https for URL (no change, same error as above)

different user-agents: (iPhone, Chrome for Windows, Safari for MacOS) (no change same error as above)

random user agent using growattServer.GrowattApi(True) (no change same error as above).

Creating a new growatt user account on my main account and using those credentials instead

Doing some quick google research it seems other libraries for pulling Growatt data are also having the same issue

Login problem PLCHome/growatt#21

Growatt integration sensors unavailable home-assistant/core#100874

Geen verbinding met Growatt server DiedB/Homey-SolarPanels#303

Hopefully this helps someone with a bit more experience with python to fix the issue

To add to this, and why i believe it's Growatt messing up bot detection: https://github.com/home-assistant/core/issues/100874#issuecomment-1741710004 when navigating to server-api.growatt.com by browser

ryandrake182 commented 9 months ago

@indykoning You're welcome, I tried to figure it out but got nowhere, I'm not much of a developer though 🤣

For what it's worth, for me the ShinePhone app is working, although I'm on iOS. It may be that just the android app is having issue but sadly I don't have an android device to test with.

From your screenshot I would agree their WAF is incorrectly configured, it might be that they fix this in a few days when they realise the android app is broken. I will try to raise issue of the broken android app with their support in the hopes they fix their side!

APKing1 commented 9 months ago

FYI - ShinePhone App working fine on my Android devices right now.

Sweenylein commented 9 months ago

Growatt has just released a ShineApp update for the iPhone. Optimise Login Logic

I think there was a server change, the apps had to be adjusted.

The third-party systems are excluded for now.

vtraveller commented 9 months ago

ah @indykoning - you found my https://github.com/home-assistant/core/issues/100874#issuecomment-1741710004

FWIW I think that the newTwoLoginAPI.do has been expanded out.

view-source:https://server.growatt.com/login in the login2() function. I'm not sure there's a radical change in the behaviour of the backend. The main login page files have just been rejigged.

I'm kinda busy in the week, so I've not really had the time to deep dive, but my investigations were going to centralize on that, as I think there just needs to be a tweak in this repo around init.py:131 to work against the new server code.

gconceicao commented 9 months ago

It is seem that login url change and cookies is a input as well.

POST /newTwoLoginAPIV2.do HTTP/1.1 Cookie: acw_tc="0bc1a08716965472817346460e7c421aad93a714ec95ecddfb1c9dd7797124";$Path="/";$Domain="server-api.growatt.com"; JSESSIONID=01F3E0091E4D2EA4FB278D6316E69F4D; assToken=bececd42fc2f24fda063343e48caac61; SERVERID=f3dd01374e65b00d703f65f037a1144c|1696548335|1696548333 Content-Type: application/x-www-form-urlencoded;charset=UTF-8 Content-Length: 335 User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2102J20SG Build/TKQ1.221013.002) Host: server-api.growatt.com Connection: Keep-Alive Accept-Encoding: gzip

vtraveller commented 9 months ago

It is seem that login url change and cookies is a input as well.

POST /newTwoLoginAPIV2.do HTTP/1.1

Cookie: acw_tc="0bc1a08716965472817346460e7c421aad93a714ec95ecddfb1c9dd7797124";$Path="/";$Domain="server-api.growatt.com"; JSESSIONID=01F3E0091E4D2EA4FB278D6316E69F4D; assToken=bececd42fc2f24fda063343e48caac61; SERVERID=f3dd01374e65b00d703f65f037a1144c|1696548335|1696548333

Content-Type: application/x-www-form-urlencoded;charset=UTF-8

Content-Length: 335

User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2102J20SG Build/TKQ1.221013.002)

Host: server-api.growatt.com

Connection: Keep-Alive

Accept-Encoding: gzip

Yes. That was my conclusion too. My starting point for fixing this was to walk the web version. I think the Android app was using the same services and they all got rejigged.

You can see from the website this isn't a well honed code base but a simple series of hacked up files meeting various product needs.

The changes seem to have moved code, but also added support for some unreleased EV charger product, which has its own server to access data from.

I don't think it's impossible to fix. Just that the new code needs to be unfurled. I was going to look at the WayBackMachine to see what was in the original /newTwoLoginAPIV2.do file, so I can piece it all back together.

ghost commented 9 months ago

If you could fix it that would be absolutely brilliant. The sole purpose of my Home Assistant installation is to link my Growatt system to my immersion (hot water tank) heater. I am not able to use Grott as my installer monitors my system and after checking with them they say that any alteration of the WiFi dongle will invalidate my service agreement.

muppet3000 commented 9 months ago

I'm really interested in a fix for this, mainly because I spent a few hours the other night looking at the new API and the netSSL output etc and I've made a carbon copy of what the app does and I still get the error so I can't work out what it does!

Sweenylein commented 9 months ago

@muppet3000

Did you try this? https://github.com/indykoning/PyPi_GrowattServer/commit/35d50c42217b340de4596877e743a231a8859c16

muppet3000 commented 9 months ago

@muppet3000

Did you try this? https://github.com/indykoning/PyPi_GrowattServer/commit/35d50c42217b340de4596877e743a231a8859c16

No, I was exclusively trying to replicate the behaviour of the app. Does that change work?

Sweenylein commented 9 months ago

@muppet3000

For the registration via https://server.growatt.com/login I get - reproduced in Postman - at least once {"result":1} and PlantID etc. back. However, I do not manage to retrieve any more data.

Step 1: https://server.growatt.com/login?account=xxxx&passwordCrc=yyyy&login_type="web" with User-Agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.81"

vtraveller commented 9 months ago

Useful info. There's a lot of redundant pathways threaded through the HTML and scripts. Oss list retrieval is quite prolific, but I'm thinking mainly irrelevant things for our goal.

My plan today:

check WBM to see if twoLogin has just been moved to a layer
graph the crazy amount of regional/product growatt servers listed in-code
graph the ancillary helper code the pages pull in
auto translate the code comments from Chinese
model the login sequence and check for cookies vs token passing on post requests
swap out code in the growattServer python after understanding the failure points inactivity debugging

I'm not specifically looking at this as a 'web' problem, but more a bunch of growatt engineers juggled code around while prepping for their ev product. Because that's what it feels like they did.

I see no evidence they've improve their software engineering skills. Just bolted more in while rejigging things.

But hey. That's just my gut speaking. I could well be wrong. I'm just hoping it doesn't take days to work out.

muppet3000 commented 9 months ago

It's worth remembering that this library was reverse engineered from the Android API, which I assume is reused by the web UI as well. At this point I've almost completely removed my need for interfacing by the API for my personal usage, however a lot of the open source projects I support(ed) use this library so it would be nice to have a fix to get a little bit more life out of them.

I agree with @vtraveller - It's more likely that the recent changes are as a result of introducing new logic rather than intentionally trying to deny access for us users. That said, we have seen a clamp down over the last 12 months in various things e.g. they definitely introduced rate limiting and IP blocking, they also changed the 'user agent' logic to do some form of verification at login.

The thing I can't get my head around is what they've changed this time, I tried to replicate it exactly the same as the packets were showing in netssl (which is essentially wireshark for android at this point) and I still couldn't get it to let me in. It's almost as if they've found a way of identifying that it's coming from a python client rather than from their app.

Sweenylein commented 9 months ago

I tried this here: https://github.com/PLCHome/growatt (Node.js)

With a key instead of username/password it works.

vtraveller commented 9 months ago

Well it was a journey but I've cracked it now. Basically any dead end, and it'll lie with its responses claiming you're not logged in.

My conclusion here is that, at some level they're actually trying to make the data servers better (yeah, really). So there's more structure now.

The above PLCHome/growatt repo has changes that match what I found out the hard way.

Oh well. My goal is actually rule control, because the supported ones are kinda dumb if you want to discharge a battery to a level, but only into the house, not the grid. I dynamically alter battery charge/discharge levels based on weather and tariff rates. I consider when to heat water.

{
  "result":1,
  "obj":
  {
    "currPage":1,
    "pages":1,
    "pageSize":4,
    "count":1,
    "ind":1,
    "datas":[
      {
        "ptoStatus":"0",
        "timeServer":"2023-10-07 20:37:11",
        "accountName":"xxxxxx",
        "timezone":"1",
        "bctMode":"0",
        "bdcStatus":"0",
        "eMonth":"78.4",
        "dtc":"3701",
        "pac":"-2564.1",
        "batSysRateEnergy":"13.0",
        "datalogSn":"JPCxxxxxx",
        "alias":"WPDxxxxxx",
        "sn":"WPDxxxxxx",
        "deviceType":"0",
        "plantId":"xxxxxxx",
        "deviceTypeName":"spa",
        "nominalPower":"3000",
        "eToday":"7.8",
        "datalogTypeTest":"ShineWIFI-S",
        "eTotal":"2809.5",
        "showDeviceModel":"SPA1~3k TL BL",
        "location":"","deviceModel":
        "SPA3000TL BL",
        "plantName":"XXXXXXXX",
        "status":"6",
        "lastUpdateTime":"2023-10-07 13:37:11"
      }
    ],
    "notPager":false
  }
}

J4CE2 commented 9 months ago

I tried this here: https://github.com/PLCHome/growatt (Node.js)

With a key instead of username/password it works.

How do you actually get the key though? Who do we email?

muppet3000 commented 9 months ago

Well it was a journey but I've cracked it now. Basically any dead end, and it'll lie with its responses claiming you're not logged in.

My conclusion here is that, at some level they're actually trying to make the data servers better (yeah, really). So there's more structure now.

The above PLCHome/growatt repo has changes that match what I found out the hard way.

Oh well. My goal is actually rule control, because the supported ones are kinda dumb if you want to discharge a battery to a level, but only into the house, not the grid. I dynamically alter battery charge/discharge levels based on weather and tariff rates. I consider when to heat water.
{
  "result":1,
  "obj":
  {
    "currPage":1,
    "pages":1,
    "pageSize":4,
    "count":1,
    "ind":1,
    "datas":[
      {
        "ptoStatus":"0",
        "timeServer":"2023-10-07 20:37:11",
        "accountName":"xxxxxx",
        "timezone":"1",
        "bctMode":"0",
        "bdcStatus":"0",
        "eMonth":"78.4",
        "dtc":"3701",
        "pac":"-2564.1",
        "batSysRateEnergy":"13.0",
        "datalogSn":"JPCxxxxxx",
        "alias":"WPDxxxxxx",
        "sn":"WPDxxxxxx",
        "deviceType":"0",
        "plantId":"xxxxxxx",
        "deviceTypeName":"spa",
        "nominalPower":"3000",
        "eToday":"7.8",
        "datalogTypeTest":"ShineWIFI-S",
        "eTotal":"2809.5",
        "showDeviceModel":"SPA1~3k TL BL",
        "location":"","deviceModel":
        "SPA3000TL BL",
        "plantName":"XXXXXXXX",
        "status":"6",
        "lastUpdateTime":"2023-10-07 13:37:11"
      }
    ],
    "notPager":false
  }
}

So what were the changes? Do you think they can be applied to this library? I'm mainly interested because I couldn't work it out myself!

gconceicao commented 9 months ago

I am trying to fixed it as weel. But for now without success.

I tried to add a cookies and add a assToken inside a cookies. but the server is respond with 501.

` resp = self.session.post(self.get_url("/")) cookies = requests.utils.dict_from_cookiejar(self.session.cookies)

    assToken = secrets.token_hex(16)
    cookies['assToken']=assToken
    self.session.cookies.update(cookies)`

I edit the post login to APIV2 as well.

Sweenylein commented 9 months ago

@J4CE2

it‘s the Share Key Link. It is a link that is automatically sent to you. Log in via a browser on your computer and apply for it

J4CE2 commented 9 months ago

@J4CE2

it‘s the Share Key Link. It is a link that is automatically sent to you. Log in via a browser on your computer and apply for it

Login where Apply where? Here? https://server.growatt.com/ Please send me the URL to apply.

Sweenylein commented 9 months ago

@J4CE2

yes, Server.growatt.com

There under energy, plant management, operating tools you can send yourself a key (Share) by e-mail.

vtraveller commented 9 months ago

Well I went down an entire annex with cookies, but that was only part of the story. The trick is to think like a Growatt Engineer, and fragment a reasonable idea into a collection of ideas 🤣

don't hash the password response = self.session.post(self.get_url('login'), data={ 'account': username, 'password': password })
grab the PlantId from a cookie return self.session.cookies.get('onePlantId')

the json data is accessed through a heady mix of query string and form data

response = self.session.post(self.get_url('panel/getPlantData?plantId=<plantId>'))
response = self.session.post(self.get_url('panel/getDevicesByPlantList'), params = {
      'currPage': '1',
      'plantId': '<plantId>'
    })
response = self.session.post(self.get_url('panel/getDevicesByPlant?plantId=<plantId>'))
response = self.session.post(self.get_url('panel/spa/getSPAStatusData?plantId=<plantId>'), params = {
        'spaSn': 'WPDxxxxxx'})

Here's all the gets I've found:

https://server.growatt.com/index/getPlantListTitle
https://server.growatt.com/index/getChuanghuoDeviceList
https://server.growatt.com/index/getPlantTopic + formData { plantId }
https://server.growatt.com/homeEnergy/getIsEicUser + formData { plantId }
https://server.growatt.com/components/getHaveNeo + formData { plantId }
https://server.growatt.com/panel/getDevicesByPlantList + formData { currPage: 1 + plantId }
https://server.growatt.com/panel/getDevicesByPlant?plantId=xxxxxx
https://server.growatt.com/layout/getIsLayoutType?plantId=xxxxxx
https://server.growatt.com/panel/getPanelPageByType?ttt=xxxxxxxx  + formData { type: spa + sn: WPDxxxxxxx }
https://server.growatt.com/index/getWeatherByPlantId?plantId=xxxxxxx
https://server.growatt.com/panel/spa/getSPATotalData?plantId=xxxxxxx + formData { spaSN: WPDxxxxxxx }
https://server.growatt.com/panel/spa/getSPAStatusData?plantId=xxxxxxxx + formData { spaSN: WPDxxxxxxx }
https://server.growatt.com/panel/spa/getSPABatChart + formData { plantId: xxxxxxxx + spaSn: WPDxxxxxx + [date: 2023-10-07] }
https://server.growatt.com/panel/getEicDevicesByPlant + formData { plantId }
https://server.growatt.com/panel/getEicPanelPageByType?type=spa&sn=WPDxxxxxx
https://server.growatt.com/panel/spa/getSPAEnergyDayChart + formData { date: 2023-10-07 + plantId: xxxxxxx spaSn: WPDxxxxx }
https://server.growatt.com/panel/getPlantData?plantId=xxxxxxxx
https://server.growatt.com/panel/spa/getSPAEnergyDayChart + formData { plantId: xxxxxxxx + spaSn: WPDxxxxxx + date: 2023-10-07 }
https://server.growatt.com/panel/getDeviceInfo + formData { plantId: xxxxxxx + deviceTypeName: datalog + sn: JPCxxxxxx

Right - now on to the set rules.

vtraveller commented 9 months ago

okay - set was easier

https://server.growatt.com/commonDeviceSetC/setSpa?type=server&spaSn=WPDxxxxxxx&ttt=xxxxxxxxxx
https://server.growatt.com/tcpSet.do + formData {
action: spaSet
serialNum: WPDxxxxxxx
type: spa_ac_charge_time_period
param1: 100
param2: 69
param3: 01
param4: 00
param5: 08
param6: 00
param7: 1
param8: 20
param9: 00
param10: 20
param11: 00
param12: 1
param13: 00
param14: 00
param15: 00
param16: 00
param17: 1
}

muppet3000 commented 9 months ago

Well I went down an entire annex with cookies, but that was only part of the story. The trick is to think like a Growatt Engineer, and fragment a reasonable idea into a collection of ideas 🤣

don't hash the password response = self.session.post(self.get_url('login'), data={ 'account': username, 'password': password })
grab the PlantId from a cookie return self.session.cookies.get('onePlantId')

the json data is accessed through a heady mix of query string and form data

response = self.session.post(self.get_url('panel/getPlantData?plantId=<plantId>'))
response = self.session.post(self.get_url('panel/getDevicesByPlantList'), params = {
      'currPage': '1',
      'plantId': '<plantId>'
    })
response = self.session.post(self.get_url('panel/getDevicesByPlant?plantId=<plantId>'))
response = self.session.post(self.get_url('panel/spa/getSPAStatusData?plantId=<plantId>'), params = {
        'spaSn': 'WPDxxxxxx'})

Here's all the gets I've found:

https://server.growatt.com/index/getPlantListTitle
https://server.growatt.com/index/getChuanghuoDeviceList
https://server.growatt.com/index/getPlantTopic + formData { plantId }
https://server.growatt.com/homeEnergy/getIsEicUser + formData { plantId }
https://server.growatt.com/components/getHaveNeo + formData { plantId }
https://server.growatt.com/panel/getDevicesByPlantList + formData { currPage: 1 + plantId }
https://server.growatt.com/panel/getDevicesByPlant?plantId=xxxxxx
https://server.growatt.com/layout/getIsLayoutType?plantId=xxxxxx
https://server.growatt.com/panel/getPanelPageByType?ttt=xxxxxxxx  + formData { type: spa + sn: WPDxxxxxxx }
https://server.growatt.com/index/getWeatherByPlantId?plantId=xxxxxxx
https://server.growatt.com/panel/spa/getSPATotalData?plantId=xxxxxxx + formData { spaSN: WPDxxxxxxx }
https://server.growatt.com/panel/spa/getSPAStatusData?plantId=xxxxxxxx + formData { spaSN: WPDxxxxxxx }
https://server.growatt.com/panel/spa/getSPABatChart + formData { plantId: xxxxxxxx + spaSn: WPDxxxxxx + [date: 2023-10-07] }
https://server.growatt.com/panel/getEicDevicesByPlant + formData { plantId }
https://server.growatt.com/panel/getEicPanelPageByType?type=spa&sn=WPDxxxxxx
https://server.growatt.com/panel/spa/getSPAEnergyDayChart + formData { date: 2023-10-07 + plantId: xxxxxxx spaSn: WPDxxxxx }
https://server.growatt.com/panel/getPlantData?plantId=xxxxxxxx
https://server.growatt.com/panel/spa/getSPAEnergyDayChart + formData { plantId: xxxxxxxx + spaSn: WPDxxxxxx + date: 2023-10-07 }
https://server.growatt.com/panel/getDeviceInfo + formData { plantId: xxxxxxx + deviceTypeName: datalog + sn: JPCxxxxxx

Right - now on to the set rules.

Well you've had a fun day! I particularly enjoyed the commentary on this. So all your queries are hitting the main Growatt website instead of the android API which you have to assume has the same data on it just a case of finding it. If there are some direct mappings from the old android URLs that we used to hit and the web page ones that would be nice but at this point I'm just interested. Personally I've switched to using Grott and I think that's my future as it removes dependency on the Growatt servers which are always flaky.

vtraveller commented 9 months ago

Can you set stuff with Grott? It's never been clear to me if you can dynamically control the rules.

(I'm just mapping my code over, but yeah, all the labels have changed)

muppet3000 commented 9 months ago

Can you set stuff with Grott? It's never been clear to me if you can dynamically control the rules.

(I'm just mapping my code over, but yeah, all the labels have changed)

The labels are all the same in the netssl output though so you're just hitting a completely different set of URLs than this library originally worked against, but working is better than not working right?

Re: Grott - Yes, sort of, you can run grottserver which pretends to be the server and then you can send write commands through that. You have to know what registers to write to though, but the Grott developer has started documenting those. I'm planning on writing a python library that can be used with it. I only set it up for the first time this week but I've got it configuring my overnight charge amount and rate which is a nice start. Between me on the Home Assistant side and the guy that made Grott I'm hoping we can make a nice solution that has no reliance on the ever-changing, unsupported Growatt APIs. My end-goal is to submit a PR to Grott which allows it to publish to both the Growatt servers (so the app still works) and the Grott server (so when the Growatt servers are down i still get data and can do config easily). My first priority though is to update all the documentation on my integrations and how to setup Grott because that's what everyone has been complaining about the most since the Apis changed this week.

vtraveller commented 9 months ago

I look forward to an alt, although changing the registers directly is somewhat scary given the numerous variants of Growatt out there, and the ability to alter everything including the voltages in the system. 😱 If it works offline, then Grott would hands down win.

It's interesting to see how they're moving the code on the web though. In theory there's more historical data than would be held locally, but then again, a solid API and gathering any additional archive sources (if available) is quite appealing. A lot of the webserver is baggage to allow installers to act as support and monitor large numbers of systems - so there's also plenty to declutter.

tbh at this point, I'm just glad they've kept to their old ways, and just cleaned up a few bits. There were many tricks they could have used to lock 3rd parties, and thankfully they've not done that. Then again, security isn't really their strong suit either - a rolling date as a password, is barely a password.

J4CE2 commented 9 months ago

@J4CE2

yes, Server.growatt.com

There under energy, plant management, operating tools you can send yourself a key (Share) by e-mail.

Thanks, got the key/url

vtraveller commented 9 months ago

In Conclusion

For my needs I've distilled everything down to a simple set of summary APIs, as I don't need graphing or historical logs at the moment. tbh there's a lot of junky duplicate data in the dataset, related to a mix of multiple plant and device management.

As I have a SPA3000, I can only easily see that device and how it works. My intent is to create a lightweight API very similar to the original GrowattServer.py, but it won't be backward compatible ... and I don't think it should be.

I understand now what Growatt did, and how things work. If there is a future break, and they don't start to crackdown on home use (of our own stuff), I think it'll be straightforward to adapt.

This is all I need for my control s/w. I need to have the current SOC, chargePower, pdisCharge1, the day summary, and the ability to set rules on/off as needed. If that helps anyone else, I'm pleased.

https://server.growatt.com/login + { 'account': username, 'password': password }
login -> access 'onePlantId' cookie = 1234567

https://server.growatt.com/panel/getEicDevicesByPlant + { plantId: 1234567 }
{
  "result":1,
  "obj":
  {
    "sn":"WPDxxxxxxxxxxx",
    "panelType":"spa"
  }
}

https://server.growatt.com/panel/spa/getSPAStatusData?plantId=1234567 + { spaSn: WPDxxxxxxxxxxx }
{
  "result":1,
  "obj":
  {
    "pdisCharge1":0.0,
    "chargePower":2.09,
    "uwSysWorkMode":"6",
    "upsVac1":"0",
    "SOC":"85",
    "wBatteryType":"1",
    "pactouser":0.0,
    "vBat":"54.1",
    "fAc":"50",
    "vac1":"248.6",
    "vAc1":"248.6",
    "priorityChoose":"0",
    "lost":"spa.status.normal",
    "upsFac":"0",
    "ppv":3.05,
    "pactogrid":0.0,
    "pLocalLoad":0.96,
    "status":"6"
  }
}

https://server.growatt.com/panel/spa/getSPATotalData?plantId=1234567 + { spaSn: WPDxxxxxxxxxxx }
{
  "result":1,
  "obj":
  {
    "eselfToday":"5",
    "gridPowerTotal":"984.8",
    "eselfTotal":"2793.7",
    "elocalLoadToday":"10.4",
    "gridPowerToday":"5.4",
    "elocalLoadTotal":"3778.5",
    "photovoltaicRevenueToday":"7.8",
    "etoGridToday":"0",
    "edischarge1Total":"1196.6",
    "photovoltaicRevenueTotal":"3370.4",
    "unit":"￡",
    "edischarge1Today":"1.4",
    "epvInverterTotal":"2808.7",
    "etogridTotal":"226.9",
    "epvInverterToday":"6.5"
  }
}

"spa_load_flast"
  loadFirstStartTime1 = postData.param1+":"+postData.param2;
  loadFirstStopTime1 = postData.param3+":"+postData.param4;
  loadFirstStartTime2 = postData.param6+":"+postData.param7;
  loadFirstStopTime2 = postData.param8+":"+postData.param9;
  loadFirstStartTime3 = postData.param11+":"+postData.param12;
  loadFirstStopTime3 = postData.param13+":"+postData.param14;
  loadFirstSwitch1 = postData.param5;
  loadFirstSwitch2 = postData.param10;
  loadFirstSwitch3 = postData.param15;
"spa_ac_discharge_time_period"
  forcedDischargeTimeStart1 = postData.param3+":"+postData.param4;
  forcedDischargeTimeStop1 = postData.param5+":"+postData.param6;
  forcedDischargeTimeStart2 = postData.param8+":"+postData.param9;
  forcedDischargeTimeStop2 = postData.param10+":"+postData.param11;
  forcedDischargeTimeStart3 = postData.param13+":"+postData.param14;
  forcedDischargeTimeStop3 = postData.param15+":"+postData.param16;
  gridFirstSwitch1 = postData.param7;
  gridFirstSwitch2 = postData.param12;
  gridFirstSwitch3 = postData.param17;
"spa_ac_charge_time_period"
  forcedChargeTimeStart1 = postData.param3+":"+postData.param4;
  forcedChargeTimeStop1 = postData.param5+":"+postData.param6;
  forcedChargeTimeStart2 = postData.param8+":"+postData.param9;
  forcedChargeTimeStop2 = postData.param10+":"+postData.param11;
  forcedChargeTimeStart3 = postData.param13+":"+postData.param14;
  forcedChargeTimeStop3 = postData.param15+":"+postData.param16;
  batFirstSwitch1 = postData.param7;
  batFirstSwitch2 = postData.param12;
  batFirstSwitch3 = postData.param17;
"spa_ac_discharge_time_period1"
  forcedDischargeTimeStart4 = postData.param1+":"+postData.param2;
  forcedDischargeTimeStop4 = postData.param3+":"+postData.param4;
  forcedDischargeTimeStart5 = postData.param6+":"+postData.param7;
  forcedDischargeTimeStop5 = postData.param8+":"+postData.param9;
  forcedDischargeTimeStart6 = postData.param11+":"+postData.param12;
  forcedDischargeTimeStop6 = postData.param13+":"+postData.param14;
  forcedDischargeStopSwitch4 = postData.param5;
  forcedDischargeStopSwitch5 = postData.param10;
  forcedDischargeStopSwitch6 = postData.param15;
"spa_ac_charge_time_period1"
  forcedChargeTimeStart4 = postData.param1+":"+postData.param2;
  forcedChargeTimeStop4 = postData.param3+":"+postData.param4;
  forcedChargeTimeStart5 = postData.param6+":"+postData.param7;
  forcedChargeTimeStop5 = postData.param8+":"+postData.param9;
  forcedChargeTimeStart6 = postData.param11+":"+postData.param12;
  forcedChargeTimeStop6 = postData.param13+":"+postData.param14;
  forcedChargeStopSwitch4 = postData.param5;
  forcedChargeStopSwitch5 = postData.param10;
  forcedChargeStopSwitch6 = postData.param15;

https://server.growatt.com/tcpSet.do +
{
  action: spaSet
  serialNum: WPDxxxxxxxxxxx
  type: spa_ac_charge_time_period
  param1: 100
  param2: 69
  param3: 01
  param4: 00
  param5: 08
  param6: 00
  param7: 1
  param8: 20
  param9: 00
  param10: 20
  param11: 00
  param12: 1
  param13: 00
  param14: 00
  param15: 00
  param16: 00
  param17: 1
}

{
  "msg":"inv_set_success",
  "success":true
}

ghost commented 9 months ago

Is this amazing work going to translate into a new working Home Assistant Integration. If yes may be so rude and ask what sort of timescale you are looking at??

vtraveller commented 9 months ago

Is this amazing work going to translate into a new working Home Assistant Integration. If yes may be so rude and ask what sort of timescale you are looking at??

I personally won't be integrating into Home Assistant. Nor will I be changing PyPi_GrowattServer.

I thought about this for quite a while, and concluded that the changes were too thorny as it would mean everyone using it would need to adapt their layers.

Instead, I'm creating my own python equivalent, but using my ideas on the type of API I want to use. I've actually completed the first cut of the this.

It uses a server list, which can be overridden, and tunes the TTL cache to half the data logger interval. It also has a low level raw API, and a high level access API. The idea being that the high level one shouldn't need to change when growatt rejig things. The low level one returns objects, which the client then has to deal with.

I'm undecided on whether to make the code public, basically because it won't support a Mix device, more than one plant, or multiple devices. I made those decisions as I don't have equipment I can test against.

If there's enough interest, I don't mind releasing it, but I don't want to be caught on a heavy support burden either.

vtraveller commented 9 months ago

@vtraveller

Have you already found a way to read out electricity production for a day? I'm already trying 3 hours.

Which one do you want, the graph or the summary?

https://server.growatt.com/panel/spa/getSPATotalData?plantId=1234567 + { spaSn: abcdefg }
{
  "result":1,
  "obj":
  {
    "eselfToday":"5",
    "gridPowerTotal":"984.8",
    "eselfTotal":"2793.7",
    "elocalLoadToday":"10.4",
    "gridPowerToday":"5.4",
    "elocalLoadTotal":"3778.5",
    "photovoltaicRevenueToday":"7.8",
    "etoGridToday":"0",
    "edischarge1Total":"1196.6",
    "photovoltaicRevenueTotal":"3370.4",
    "unit":"￡",
    "edischarge1Today":"1.4",
    "epvInverterTotal":"2808.7",
    "etogridTotal":"226.9",
    "epvInverterToday":"6.5"
  }
}

You can try adding: date: 2023-10-07 to the form data for a different day. Might work the same way as the chart data.

ghost commented 9 months ago

Is this amazing work going to translate into a new working Home Assistant Integration. If yes may be so rude and ask what sort of timescale you are looking at??

I personally won't be integrating into Home Assistant. Nor will I be changing PyPi_GrowattServer.

I thought about this for quite a while, and concluded that the changes were too thorny as it would mean everyone using it would need to adapt their layers.

Instead, I'm creating my own python equivalent, but using my ideas on the type of API I want to use. I've actually completed the first cut of the this.

It uses a server list, which can be overridden, and tunes the TTL cache to half the data logger interval. It also has a low level raw API, and a high level access API. The idea being that the high level one shouldn't need to change when growatt rejig things. The low level one returns objects, which the client then has to deal with.

I'm undecided on whether to make the code public, basically because it won't support a Mix device, more than one plant, or multiple devices. I made those decisions as I don't have equipment I can test against.

If there's enough interest, I don't mind releasing it, but I don't want to be caught on a heavy support burden either.

I am very happy to test it for you and I promise I will not ask for support unless it is vital!! :) :)

Sweenylein commented 9 months ago

@vtraveller

Have you already found a way to read out electricity production for a year? I'm already trying 3 hours. (sorry for my wrong post few minutes ago)

indykoning / PyPi_GrowattServer

405 Client Error #74

In Conclusion