search

BlazejosP / huawei-sun2000-API-CLI

Huawei SUN2000 compatible devices which working with Huawei Cloud Service command line bash API for monitoring or download data from their FusionSolarApp API
https://www.buymeacoffee.com/blazejosP
GNU General Public License v3.0
110 stars 15 forks source link

close to real time #4

Closed lwodniak closed 3 years ago

lwodniak commented 3 years ago

Hi, you can use the data from the inverter then you get 'current power' with minimal delay. thirdData / getDevRealKpi active_power ("devTypeId":"1"). It is faster than data from 'kiosk mode'

BlazejosP commented 3 years ago

Hi, you can use the data from the inverter then you get 'current power' with minimal delay. thirdData / getDevRealKpi active_power ("devTypeId":"1"). It is faster than data from 'kiosk mode'

Hi thanks for info. I think about implementing this in code. I have some question for you If you have this already implemented in script is possible that you may share here this fragment of code that I can add this and integrate inside this software?

lwodniak commented 3 years ago

I only use it for 'domoticz'. For now, I have it done quite simply. The python script inserts the data into the domoticz for daily and current use. I have a crontab set up every 2 minutes.


import requests
import json
import decimal
userName=....
systemCode=....

s = requests.Session()
s.headers.update({'Connection': 'keep-alive', "Accept": "*/*","Accept-Encoding": "gzip,deflate,sdch", 'Content-Type': 'application/json'});
r = s.post(url='https://eu5.fusionsolar.huawei.com/thirdData/login', json={"userName":userName,"systemCode":systemCode},headers=s.headers)
data=r.json()
cook= r.cookies.get(name='XSRF-TOKEN')

s.headers.update({'Connection': 'keep-alive', "Accept": "*/*","Accept-Encoding": "gzip,deflate,sdch", 'Content-Type': 'application/json', 'XSRF-TOKEN': cook})
r = s.post(url='https://eu5.fusionsolar.huawei.com/thirdData/getStationList', json={},headers=s.headers)
data=r.json()
sc=data['data'][0]['stationCode']

r = s.post(url='https://eu5.fusionsolar.huawei.com/thirdData/getDevList', json={"stationCodes":sc},headers=s.headers)
data=r.json()
devIds=data['data'][0]['id']

r = s.post(url='https://eu5.fusionsolar.huawei.com/thirdData/getDevRealKpi', json={"stationCodes":sc,"devIds":devIds, "devTypeId":"1"},headers=s.headers)
data=r.json()
dane=data.get("data")[0]
print(decimal.Decimal(dane.get("dataItemMap").get("active_power"))*1000)
print(decimal.Decimal(dane.get("dataItemMap").get("day_cap"))*1000)
url = "http://192.168.1.150/json.htm?type=command&param=udevice&idx=190&nvalue=0&svalue="+str(decimal.Decimal(dane.get("dataItemMap").get("active_power"))*1000)+';'+str(int(decimal.Decimal(dane.get("dataItemMap").get("day_cap"))*1000))
s3 = requests.Session()
r = s3.get(url=url)
BlazejosP commented 3 years ago

In my case I receive only that from https://eu5.fusionsolar.huawei.com/thirdData/getDevRealKpi no JOSN positions active_power and day_cap I have SUN2000-5KTL-M0 what inverter do you have? Still you can use my scripts to capture from your device but is necessary to change this fields in sh files.

{ "dataItemMap": { "day_power": "0.0", "month_power": "25.8", "total_power": "2412.63", "day_income": "0.0", "total_income": "311.5099", "real_health_state": "3" },

that is why I need use kiosk mode for near real time data no another option. I'm curious why your inverter has this addtional fields in JOSN response from getDevRealKpi? What model is that?

lwodniak commented 3 years ago

I have the same SUN2000-5KTL-M0 from:https://eu5.fusionsolar.huawei.com/thirdData/getDevList I get: 'data': [{ 'devName': 'HV2030014150', 'devTypeId': 1, 'esnCode': 'HV2030014150', 'id': 4555937883504, 'invType': 'SUN2000-5KTL-M0', 'softwareVersion': 'V100R001C00SPC122' ... ]

from: https://eu5.fusionsolar.huawei.com/thirdData/getDevRealKpi i get: {'dataItemMap': {'inverter_state': 2.0, 'ab_u': 404.0, 'bc_u': 402.6, 'ca_u': 404.3, 'a_u': 233.1, 'b_u': 232.1, 'c_u': 233.0, 'a_i': 0.0, 'b_i': 0.0, 'c_i': 0.0, 'efficiency': 0.0, 'temperature': 7.7, 'power_factor': 0.0, 'elec_freq': 49.95, 'active_power': 0.0, 'reactive_power': 0.0, 'day_cap': 0.0, 'mppt_power': 0.0, 'pv1_u': 211.8, 'pv2_u': 211.8, ... 'pv24_u': 0.0, 'pv1_i': 0.0, 'pv2_i': 0.0, ... 'pv24_i': 0.0, 'total_cap': 1608.61, 'open_time': 1611564700.0, 'close_time': 1611587665.0, 'mppt_total_cap': 0.0, 'mppt_1_cap': 0.0, 'mppt_2_cap': 0.0, ... 'mppt_9_cap': 0.0, 'mppt_10_cap': 0.0, 'run_state': 2} }

I got this documentation from huawei. SmartPVMS V300R006C10_API_Northbound Interface Reference (1).pdf

BlazejosP commented 3 years ago

I investigates this what you show me made some experiments and have two possible answers on question why we have different data from the same devices?

First There is difference in software version between or inverters My "softwareVersion": "V100R001C00SPC111", Your 'softwareVersion': 'V100R001C00SPC122' But I have feeling this is less likely reason

Secound When I test with Postman questions to API I found that I can only ask question with station codes. Of course that give me some data but simpler than you have. I can ask question to API until chapter 2.8 in this manual https://github.com/BlazejosP/huawei-sun2000-API-CLI/files/5871850 Rest after chapter 2.9 need devIds which I copy from my https://eu5.fusionsolar.huawei.com/thirdData/getDevList API question but I have still "success": false so that means that I can't with my account have access to half of more precise questions to API. So conclusion is that I have normal user account granted by company which installed inverter and you have a full account with access to everything.

BlazejosP commented 3 years ago

My proposition is try first this software and if you can show me screenshot from kiosk mode (of curse anonymise private data by blur). I long term if you have this better data from your inverter from chapter 2.9 and beyond we can incorporate this into fusionsloarapp.sh in longer term. I will be also interested in expanding this software to cover all possible configurations. But for now I will ask my company which installed inverter that they can grant me this better account which you have. If not if you will be interested I ask you about access to some API questions which I can't execute (of course after anonymise) them first. Probably after month of two you will have modified fusionsolarapp.sh which can work also with your case of account. As for now will be great if you can test both files kioskmode.sh and fusionsolarapp.sh and tell me what is their behaviour of course there is less data than you have but if you will be interested we can think about expansion together. A you mentioned that you use domoticz. In this software you have three options export data to -> influxDB/grafana or into Domoticz or into MQTT broker -> Domoticz. I personally use import to Grafana to present data and together in the same time import to Domoticz with use of MQTT broker. So I don't use direct import to domoticz as you in your python script.

lwodniak commented 3 years ago

unfortunately errors pop up at startup

kioskmode.sh : not foundh: 2: kioskmode.sh: : not foundh: 10: kioskmode.sh: : not foundh: 17: kioskmode.sh: : not foundh: 22: kioskmode.sh: : not foundh: 26: kioskmode.sh: : not foundh: 29: kioskmode.sh: kioskmode.sh: 116: kioskmode.sh: Syntax error: "(" unexpected (expecting "then")

sh fusionsolarapp.sh : not foundapp.sh: 2: fusionsolarapp.sh: : not foundapp.sh: 9: fusionsolarapp.sh: : not foundapp.sh: 12: fusionsolarapp.sh: : not foundapp.sh: 23: fusionsolarapp.sh: : not foundapp.sh: 26: fusionsolarapp.sh: : not foundapp.sh: 27: fusionsolarapp.sh: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 48 0 0 100 48 0 6 0:00:08 0:00:07 0:00:01 0^C

BlazejosP commented 3 years ago

moust error from this what I found errors are in place of spaces this not foundapp.sh: look inside file kioskmode.sh Ah there is instruction inside file first is necessary install all this tools on linux

sudo apt-get install jq
sudo apt-get install httpie
sudo apt-get install grep
sudo apt-get install curl

kiosk_mode_url must have link taken from your kiosk mode settings in FusionSolarApp and show_data_in_terminal=true rest related to sending data somwhere maby false for test And after this you can start this tool again Look on readme how setup kioskmode.sh this scripts not working out off box without fine tune them in 

# Configuration section
#----------------------

#----------------------
lwodniak commented 3 years ago

in the script I put "kiosk mode" address that works in the browser I am getting such an error

jq: error (at :1): Cannot index string with string "createTime" jq: error (at :1): Cannot index string with string "id" jq: error (at :1): Cannot index string with string "language" jq: error (at :1): Cannot index string with string "logoFileId" jq: error (at :1): Cannot index string with string "permissons" jq: error (at :1): Cannot index string with string "state" jq: error (at :1): Cannot index string with string "stationName" jq: error (at :1): Cannot index string with string "title" jq: error (at :1): Cannot index string with string "token" jq: error (at :1): Cannot index string with string "updateTime"

Wrong token in kiosk mode url: can't conect to Kiosk Mode check url Not posible to capture any data :(

lwodniak commented 3 years ago

I installed 'influxDB', created the database 'solar_panels' and run fusionsolarapp.sh

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 130 100 84 100 46 6 3 0:00:15 0:00:12 0:00:03 17

No Errors device working correctly jq: error (at :1): Cannot index number with string "collectTime" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "collectTime" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" jq: error (at :1): Cannot index number with string "dataItemMap" HTTP/1.1 204 No Content Content-Type: application/json Request-Id: 602d3be6-602b-11eb-8088-dca6323c4170 X-Influxdb-Build: OSS X-Influxdb-Version: 1.8.3 X-Request-Id: 602d3be6-602b-11eb-8088-dca6323c4170 Date: Tue, 26 Jan 2021 23:08:17 GMT

HTTP/1.1 204 No Content Content-Type: application/json Request-Id: 604820ef-602b-11eb-8089-dca6323c4170 X-Influxdb-Build: OSS X-Influxdb-Version: 1.8.3 X-Request-Id: 604820ef-602b-11eb-8089-dca6323c4170 Date: Tue, 26 Jan 2021 23:08:18 GMT

HTTP/1.1 204 No Content Content-Type: application/json Request-Id: 60521a36-602b-11eb-808a-dca6323c4170 X-Influxdb-Build: OSS X-Influxdb-Version: 1.8.3 X-Request-Id: 60521a36-602b-11eb-808a-dca6323c4170 Date: Tue, 26 Jan 2021 23:08:18 GMT

HTTP/1.1 204 No Content Content-Type: application/json Request-Id: 605c56c0-602b-11eb-808b-dca6323c4170 X-Influxdb-Build: OSS X-Influxdb-Version: 1.8.3 X-Request-Id: 605c56c0-602b-11eb-808b-dca6323c4170 Date: Tue, 26 Jan 2021 23:08:18 GMT

HTTP/1.1 204 No Content Content-Type: application/json Request-Id: 6065dbfb-602b-11eb-808c-dca6323c4170 X-Influxdb-Build: OSS X-Influxdb-Version: 1.8.3 X-Request-Id: 6065dbfb-602b-11eb-808c-dca6323c4170 Date: Tue, 26 Jan 2021 23:08:18 GMT

HTTP/1.1 204 No Content Content-Type: application/json Request-Id: 60767cbb-602b-11eb-808d-dca6323c4170 X-Influxdb-Build: OSS X-Influxdb-Version: 1.8.3 X-Request-Id: 60767cbb-602b-11eb-808d-dca6323c4170 Date: Tue, 26 Jan 2021 23:08:18 GMT

./fusionsolarapp.sh: line 626: power_iverted_array: bad array subscript ./fusionsolarapp.sh: line 626: hour_of_the_day_array: bad array subscript HTTP/1.1 400 Bad Request Content-Type: application/json

BlazejosP commented 3 years ago

in the script I put "kiosk mode" address that works in the browser I am getting such an error

jq: error (at :1): Cannot index string with string "createTime" jq: error (at :1): Cannot index string with string "id" jq: error (at :1): Cannot index string with string "language" jq: error (at :1): Cannot index string with string "logoFileId" jq: error (at :1): Cannot index string with string "permissons" jq: error (at :1): Cannot index string with string "state" jq: error (at :1): Cannot index string with string "stationName" jq: error (at :1): Cannot index string with string "title" jq: error (at :1): Cannot index string with string "token" jq: error (at :1): Cannot index string with string "updateTime"

Wrong token in kiosk mode url: can't conect to Kiosk Mode check url Not posible to capture any data :(

In case of kioskmode.sh Looks that is some problem with jq or what is most probable your installation has different answer in kiosk mode when we ask this https://eu5.fusionsolar.huawei.com/kiosk/checkKioskToken . I have question would you like give me temporary access for day or two to your kioskmode. For that I don't need any login or password just link to your kioskmode. After day or two when I figure out what is different you switch off kioskmode and when you switch on again this letter them there will be different token so your data will be inaccessible after this short period. I d'like to see what is this difference in API answer between this what is in my installation and yours.

lwodniak commented 3 years ago

Write when you are done.

BlazejosP commented 3 years ago

Already done there was error in between lines 105 to 107 in kioskmode.sh I simple made wrong assumption that any token starts always with digits (what was not true in your case yours starts with letter) Now this part of code is improved and can accept anything letter or digit. Kioskmode in my case is working and puling your data. Test this on your computer. I modified kioskmode.sh on github just simple copy this new version of file on your computer. And don't forgotten switch off and on kioskomode in Huawei webpage to create new token for security this old stops to work then.

If data are appears on terminal you can in configuration send them either to your domoticz (is necessary to create first dummy Sensors and write idx inside kioskomde.sh or to influxDB because from this what I saw on logs you have working influx already if you have also Grafana you can also use included visualisation)

lwodniak commented 3 years ago

that's how it works now

BlazejosP commented 3 years ago

that's how it works now

Yeeah :) I'm really happy and that's helps for improve code of this application. I'm really appropriate your help. Error was hard to discover I test two to three tokens myself when I wrote this and by accident they were always with digits on start so this mistake escape me. For now meaby try to pull data automatically with cron and send them to domoticz sensors. You will have more data in domoticz than now. I also thinking about this second file fusionsolarapp.sh which pull data from official API. Error from your logs in this second file looks quite similar. I'm writing now the modified version of this file which will have possibility not only to send data to InfluxDB but like kioskmode.sh also to Domoticz and MQTT. So if you are intrested to have this secound file working in your installation meaby we should talking on some private chanell. Because there are usernames and passwords. See you until tomorrow morning I start to investigate second file.

lwodniak commented 3 years ago

If I didn't find your code, I wouldn't know how to get to this data and who to write to for access, , so we're even ;)

BlazejosP commented 3 years ago

Would you like to test new rewritten version of fusionsolarapp.sh you can download from this github and after inserting inside your username and password checking if is connecting to your account or not and if are any errors. That is all what is doing now + shows some data from plant list.

lwodniak commented 3 years ago

it works: src

BlazejosP commented 3 years ago

Would you like to test fusionsolarapp.sh now working with all the functions and now extract all the data from plants which I have aces. Based on your document: 2.1 Login Interface 2.2 Power Plant List Interface 2.3 Interface for Real-time Plant Data 2.4 Interface for Hourly Plant Data 2.5 Interface for Daily Plant Data 2.6 Interface for Monthly Plant Data 2.7 Interface for Yearly Plant Data 2.8 Device List Interface

are working now. Sending data to influxDB,domoticz and MQTT I make in a while back working but now I thinking about your more extended version. I haven't access to the data what you have and are not available from my installation. If would you have this also incorporated I will need some help from your side or we can cooperate together to have this part of question also done? Meaby you can implement this based on my code and help.

2.9 Interface for Real-time Device Data 2.10 Interface for 5-minute Device Data 2.11 Interface for Daily Device Data 2.12 Interface for Monthly Device Data 2.13 Interface for Yearly Device Data 2.14 Device Switch Interface . 2.15 Device Upgrade Interface 2.16 Device Upgrade Record Interface 2.17 Device Alarm Interface 2.18 SN Registration Query Interface Tell me if you encounter some problem with actually code hope that in your installation is everything working.

lwodniak commented 3 years ago

What you did works fine. Unfortunately, I do not have time now, and I did not write anything in the scripts before, as you do, so at this moment I am not able to do it myself.

BlazejosP commented 3 years ago

What you did works fine. Unfortunately, I do not have time now, and I did not write anything in the scripts before, as you do, so at this moment I am not able to do it myself.

That's ok just if you found time test this and send info about errors that's all. I'm rewriting now fusionsollarapp.sh so there will be new data from this interfaces which I tough that are related with high privilege account. Look that I have access to them also and there is no high privilege account at all so I don't need data from your account Thanks to your document from Huawei. In next 1/2 weeks I will finish data scraping interface then add possibility to send to influxdb back and Domoticz and MQTT like in kioskmode.sh