Domoticz Dummy sensortype usage

[RikBast]

Do we need to use only the dummy sensor with sensor type custom sensor? Would be nice to have also diagrams besides the graph, with sensor type electric or usage we can generate diagrams.

[MichelRabozee]

Personally, I modified a bit the code in order to send data to a virtual sensor "Electric (Instant + Counter)": it requires 2 values: an instant value (in Watts) and a total value (in WattsHour); I also have a generic (simple) custom virtual sensor for "today production" and another one for "total production".

To be able to retrieve the instant power generated, the code needs to call the "getDevRealKpi" API call (hence, you need to have the "inverter" device ID (you may get that with a "getDevList" API call, the inverter is the device with type "38"), and to call "getStationRealKpi" for the total power produced value.

You need the code to cope with "too many API calls in a small amount of time", amount of time which is quite ramdom for the getDevRealKpi call (varies from max 1 call every minute to max 1 call every 8 minutes, go figure...).

I attach the "HuaweiSolar.env" and "HuaweiSolar_API.sh" I modified. What I use for the "Electric (Instant + Counter)" is "HuaweiSolar_API.sh InverterRealTime".

HuaweiSolar_API.sh.txt HuaweiSolar.env.txt

Here is my crontab:

*/4   *    *    *    *     HuaweiSolar_API.sh InverterRealTime >/dev/null 2>&1
7     *    *    *    *     HuaweiSolar_API.sh Hour     >/dev/null 2>&1
8     0    *    *    *     HuaweiSolar_API.sh Day      >/dev/null 2>&1
9     0    *    *    *     HuaweiSolar_API.sh Year     >/dev/null 2>&1

[RikBast]

Thx for this, it seems my "devTypeId": 1 for the Inverter and the is "id": , that's the one I need for the .env file? But how to export to Domoticz, is there an extra entry needed in the HuaweiSolar.env File to refer to the DomoticzID?

And I can't get to refresh the token anymore

My inverter has an ID = 1000000035294540.

Normally, calling "HuaweiSolar_API.sh Devices" should give you the info, like this:

Device Type: 62, Device ID: 1000000035294539, Device name: "Dongle-1"
Device Type: 38, Device ID: 1000000035294540, Device name: "Inverter-1"

Regarding the token, this is the code I have (and which works):

#!/bin/bash

mydir=`dirname $0`
myTempFile=$mydir/results.$$
myTokenFile=$mydir/last.token
source $mydir/HuaweiSolar.env

curl -i -s -X POST -H 'Content-Type:application/json' -d "{\"userName\":\"$userName\",\"systemCode\":\"$systemCode\"}"  $baseURL/login >$myTempFile
export myToken=$(cat $myTempFile|grep token|cut -d ' ' -f 2)

echo $myToken
echo $myToken > $myTokenFile
cat $myTokenFile
rm $mydir/*.$$

[RikBast]

Running that command gives this error: ./HuaweiSolar_API.sh: line 73: gdate: command not found

But I do get a list of devices.

[MichelRabozee]

Oh, yes, gdate is gnu date, required to be able to handle the "+%s%3N" format to get milliseconds. I am on MacOS, and the builtin "date" is not the GNU one. If your system is like a Raspberry, just remove the 'g' of gdate.

[RikBast]

Ok. One last issue: how to get the data to domoticz? In the .env file there is no entry to refer to the DomoticzID sensor. For the Electric sensor I mean

[MichelRabozee]

Oops, right, I included an old .env file :-( I updated my message above. And please take again the "HuaweiSolar_API.sh" as well, I corrected it as per your comments about gdate.

The ID for the Electrical virtual sensor must be filled in here

# Domoticz ID for variable real time/total production
DomoticzHuaweiGeneratedRealTime=ChangeMe

[RikBast]

I will change it, I noticed after 25 min you cannot get a new token anymore with the new .sh file. The old script refreshed the token with no issue

[MichelRabozee]

If you run the get_token.sh script by hand, what is happening ?

[RikBast]

get_token.sh: 6: get_token.sh: source: not found

I run command: sh get_token.sh

Hope that is the correct way

[MichelRabozee]

get_token.sh and HuaweiSolar.env must be executable (chmod +x ) and in the same directory.

./get_token.sh should run the script.

If it runs, maybe comment out the line "rm $mydir/.$$" and look at the created "results." files and the last.token file

[RikBast]

It generated 3 results files:

Seems a 0 length token?

{ "result" : [ { "LastUpdate" : "2022-09-25 21:54:05", "Name" : "Huawei_XSRF_token_epoch", "Type" : "2", "Value" : "1664135645", "idx" : "1" } ], "status" : "OK", "title" : "GetUserVariable"

HTTP/1.1 200 OK Server: product only Date: Sun, 25 Sep 2022 21:00:01 GMT Content-Length: 0 Connection: keep-alive

HTTP/1.1 200 OK Server: product only Date: Sun, 25 Sep 2022 21:00:06 GMT Content-Length: 0 Connection: keep-alive

[MichelRabozee]

No "last.token" file in the directory ?

[RikBast]

There is last.token file in the same directory. It’s empty, maybe it also needs to be executable?

[RikBast]

[MichelRabozee]

There is last.token file in the same directory. It’s empty, maybe it also needs to be executable?

no, last.token does not have to be executable.

Just to be sure: you have put the new lines from "HuawaiSolar.env.txt" into "HuaweiSolar.env", correct ? (the files I have put here had the ".txt" extension because it is not allowed to upload ".sh" or ".env" files).

The file sizes I have are: HuaweiSolar_API.sh => 9861 get_token.sh => 409

Can you try to execute "get_token.sh" in debug mode ? use: bash -x get_token.sh

[MichelRabozee]

I also see in your screen shot that some results are 218 bytes, other 121. If all were the results of the get_token script, the ones with 121 bytes should be the ones notifying an error (without XSRF token), but the ones with 218 bytes should contain the XSRF token.

[RikBast]

There is last.token file in the same directory. It’s empty, maybe it also needs to be executable?

no, last.token does not have to be executable.

Just to be sure: you have put the new lines from "HuawaiSolar.env.txt" into "HuaweiSolar.env", correct ? (the files I have put here had the ".txt" extension because it is not allowed to upload ".sh" or ".env" files).

Yes, I manually edited the .txt file and renamed it to .env

The file sizes I have are: HuaweiSolar_API.sh => 9861 --> my Device ID = 1 instead of 38

get_token.sh => 409

Can you try to execute "get_token.sh" in debug mode ? use: bash -x get_token.sh

I get all the data, can't post it here (You will then have full access to my API account etc). So I put only the last part. myToken seems empty

• export MyToken =

• myToken =

• echo

• echo

• cat ./last.token +rm ./results.30193

[RikBast]

This is the results file. Old token, it will not update

{ "result" : [ { "LastUpdate" : "2022-09-25 21:54:05", "Name" : "Huawei_XSRF_token_epoch", "Type" : "2", "Value" : "1664135645", "idx" : "1" } ], "status" : "OK", "title" : "GetUserVariable"

[MichelRabozee]

That result file is the update of the Domoticz Huawei_XSRF_token_epoch variable, not one from get_token.

For instance, this is a result file from a successful get_token execution.

HTTP/1.1 200 OK
Server: product only
Date: Mon, 26 Sep 2022 09:24:01 GMT
Content-Type: application/json;charset=UTF-8
Content-Length: 68
Connection: keep-alive
x-trace-enable: false
x-frame-options: SAMEORIGIN
x-download-options: noopen
x-parent-id: 8726933996551503978
strict-transport-security: max-age=31536000; includeSubDomains
set-cookie: XSRF-TOKEN=x-bvcavy2lqpvw7ws6vzrsbvenjwtc5hg5vzc93yardjdhjzbvk8qodio8dhhcvs6n9f7tjsc6bsir9ejzmkrwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx;Path=/;Secure
x-trace-id: 0
x-span-id: 8726933996551503978
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
xsrf-token: x-bvcavy2lqpvw7ws6vzrsbvenjwtc5hg5vzc93yardjdhjzbvk8qodio8dhhcvs6n9f7tjsc6bsir9ejzmkrwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

{"data":null,"success":true,"failCode":0,"params":{},"message":null}

I do not have much more idea to help you :-(

[RikBast]

I can get new token via Postmaster app as suggested by Huawei manual from there support website

Running ./HuaweiSolar_API.sh InverterRealTime Y on the PI just gives me output that the token is >25 min old and it won't retrieve a new token. So strange it won't update the token

[MichelRabozee]

Maybe that is because the Huawei_XSRF_token_epoch variable in Domoticz is wrongly updated ? The HuaweiSolar_API.sh relies on the stored value of that variable in Domoticz to decide if the get_token.sh script needs to be called or not (for instance, mine was updated around 11:52 - it is now 11:58 here - and the variable contains 1664185920).

Can you check the value of the variable in Domoticz, and use an epoch converter (like: https://www.epochconverter.com ) to check the time it represents ?

[RikBast]

Will do that tonight, no acces to Domoticz at the moment.

I just used the original HuaweiSolar_API.sh and the original HuaweiSolar.env, not your edited versions and the token is retrieved with no issue. I am breaking my head here, why that is :-(

Hopefully Domoticz variable issue

[MichelRabozee]

It runs fine here, I do not see what can go wrong :-( BTW, I put in my post above a new version of the "HuaweiSolar_API.sh".

Regarding token refreshing, as far as I know, the only thing I touched in the CheckToken() mechanism is the line "myEpoch=$EPOCHSECONDS" because EPOCHSECONDS is only available in bash from version 5 (and my OS still runs bash version 3.2.57(1), hence I merely used "myEpoch=$(date +%s)" which should run everywhere.

Also, regarding InfluxDB injection, I changed

  influx -username $myInfluxUser -password $myInfluxPass -host $myInfluxHost -database $myInfluxDB -precision=ms -execute "insert $1 value=$2 $3"

with

  influx write --bucket "SolarPanels" --precision=ms "$1 value=$2 $3"

because I use InfluxDB version 2.

[RikBast]

I feel stupid...found a typo in the username in the .env file, after correction it started working fine. Sorry for all the inconvenience.

Let me try and look tonight in Domoticz at the data and see what more data we can utilize (for instance the battery)

I will have a closer look at the _API.sh file if I can edit myself for battery data. But probably need some help with this .

[MichelRabozee]

No problem, I got my share of such typos during my own tries :-) As said earlier, I put a new version of the "HuaweiSolar_API.sh" this morning (around 9:30 West European time). Beware of the InfluxDB write modification I mentioned for InfluxDB v2.

I am still waiting for the installation of the batteries (Luna 2000 10kWh), so your work will be appreciated in a very near future :-)

If it may be of use, this is the V6 API reference: iMaster NetEco V600R022C00 Northbound Interface Reference-V6 (SmartPVMS).pdf

Michel

[RikBast]

Yes, typos are so easily overlooked. I already use the new HuaweiSolar_API.sh. I don't use InfluxDB (who knows maybe something in the future)

A 10kWh battery is big! Why you chose 10 if I may ask? I have 5 kWh, and with this weather it will never be fully charged. And now with injection prices higher then retrieval prices of electricity I should sell my battery :-D

I hope I can figure out how to make it work for the battery data. thx for the API reference

Rik

[MichelRabozee]

In Belgium, since last year, the "prosumers" (the ones producing more than they consume) even have to pay a tax (we have a strange kind of ecologists here... They even make the nuclear power plants to stop and replace them with gaz turbines, because they still have "nuclear no thank you" motto from the last decades. Even the kWh reinjected to the grid are not bought by the power company, it is like we offer them that energy. Anyway. My home is quite heavily equipped with domotics, handled by computers, I have my storage on 2 NAS devices, and a pond with require constant flowing. I have a permanent power need of 800 Watts :-( I do not have much more consumption than that (only cooking, all the house is equipped with low consumption LEDs, and I do not use electricity for the heating).

With the solar panels (I have them since this September 2, 2022, with a 3.2 kWc capacity), and I already see they produce more than 2 kWh during 3-4 hours, over the 800 Wh I need, even with a cloudy weather. Hence, seeing that, I really prefer to self-consume a maximum of energy rather than offering it graciously to the energy company, and I immediately ordered the battery. Regarding the capacity, 10 kWh, if filled in, would suffice to cover the night needs, and I guess if the summers are even less sunny than the last one, the 10 kWh will be almost filled each day.

I already saw that without the battery, I consume 1/3 less than before from the grid, with the batteries, it may go to 1/2 my normal consumption at least. I expect to have a return of investment in less than 4 years.

Michel

[RikBast]

Michel, Do you also have negative values?

[MichelRabozee]

Not for several days, but I have seen that when I was using another way to get the total energy produced value (not using an openAPI account, but relying on the normal customer account). It never happened since I use the (modified) Goumjer script.

To explain what was happening with my old way to get the data: around after midnight, the value of the previous day "energy produced" was subtracted from the "total energy produced" value, leading to a negative value when the Domoticz graphing system sees that artefact. As I said, with the modified Goumjer script, this subtraction never occurred again (the values of the total energy produced keep increasing). But to check why (for next night) you may add the line

  echo `date` "Current (kW): "$myCurrentkW "kW, Current (W): "$myCurrentW "W, Total: "$myTotalWh "Wh, Today: "$myTodaykWh "kWh" >> realtime_inverter_data.txt

at the end of the "InverterRealTime ()" function, and look at the values at the time the negative data happens.

If the issue is a temporary backwards value in the "total produced energy", there is a way to cope with it by always comparing the currently read total value to the previous one and ignore a backwards value, like with a code like this:

    PreviousTotalEnergyFile="~/previous_total_energy.txt"
    # make sure the total energy value keeps growing
    if [ -r $PreviousTotalEnergyFile ]
    then
        previousTotalEnergy=`cat $PreviousTotalEnergyFile`

    if [ $previousTotalEnergy -gt $myTotalWh ]
        then
            myTotalWh=$previousTotalEnergy
        fi
    fi
    echo $myTotalWh > $PreviousTotalEnergyFile

(code to put after the line myTotalWh=$(echo "scale=0; ${myTotalkWh} * 1000 / 1" | bc | sed 's/^./0./') )

I however think this may be a one time issue due to the fact that yesterday was the first day you filled in the data for that sensor. The way the data are processed by Domoticz graphing system is still a black box to me (I use that system since 1 week only now :-) )

[MichelRabozee]

I just also saw that you defined the "Solarpanels" virtual sensor as "Usage". I think you should define it as "Return".

[RikBast]

OK, changed it to 'Return', missed that one. Let's see and wait another day.

And: the blue diagrams is generated by the PV system, the yellow line is the actual usage of power it seems, but the domotizc graph is a bit off now.

And I also live in Belgium, installed my PV end of august (12 panels@405 Wp), to cope with airco (installed a year ago) in the summer I installed 1000 kWh extra, I normally use around 3000 kWh/year. Solarpanels are due south for max performance.

I just don't know if heating with Airco will be cheaper then with gas central heating. The savings you have from electricity, you buy gas for central heating is my idea in this.

[RikBast]

I just also saw that you defined the "Solarpanels" virtual sensor as "Usage". I think you should define it as "Return".

I don't understand the zero line from 20.00 - 8.00 h, unless with 'Return' it's the actual output of the PV

[MichelRabozee]

For me, my main issue is the continuous 800 Wh (ponds, domotics), which I am trying to reduce. My central heating system is fuel (with pipes into the floors), and seems a lot cheaper than gas or even electricity (I also have inverters but I use them only as Air Conditioning, and quite seldom). My fuel consumption is less than 1 liter a day during summer (hot water production) and max 20 l./day if the external temperature goes below -10°C. Only the future evolution will say what really is the best.

[MichelRabozee]

I don't understand the zero line from 20.00 - 8.00 h, unless with 'Return' it's the actual output of the PV

The instant value (in Watts) the HuaweiSolar_API.sh script produces is the instant solar panel actual power production. I cannot see how it goes to negative values. Or maybe this is due to the batteries ? When they discharge their accumulated energy ?

[RikBast]

Another negative value.

[RikBast]

But not in the website app

![Uploading 42D3989E-4FBF-4B41-937D-3F1EB0335AA7.jpeg…]()

[RikBast]

[MichelRabozee]

Strange seems to be équivalent to the total Energy produced the previous day, and the graph goes negative because that value is temporarily (just after midnight) subtracted from the total. Did you put the line writing the data to a file to look at what is sent to domoticz ?

[MichelRabozee]

One question: the "Energy read" toggle, did you put it on "Computed" or on "From device" ? It should be "set to "From device" as the script provides the constantly increasing "Total Energy" value, no need for Domoticz to compute it.

[RikBast]

It’s set ‘From Device’

[RikBast]

Strange seems to be équivalent to the total Energy produced the previous day, and the graph goes negative because that value is temporarily (just after midnight) subtracted from the total. Did you put the line writing the data to a file to look at what is sent to domoticz ?

Will try that tonight

[MichelRabozee]

It’s set ‘From Device’

Arf, it would have been too easy if it was the issue :-)

Strange seems to be équivalent to the total Energy produced the previous day, and the graph goes negative because that value is temporarily (just after midnight) subtracted from the total. Did you put the line writing the data to a file to look at what is sent to domoticz ?

Will try that tonight

OK, to be continued :-)

[RikBast]

It’s set ‘From Device’

Arf, it would have been too easy if it was the issue :-)

Strange seems to be équivalent to the total Energy produced the previous day, and the graph goes negative because that value is temporarily (just after midnight) subtracted from the total. Did you put the line writing the data to a file to look at what is sent to domoticz ?

Will try that tonight

OK, to be continued :-)

I added the code at the end of the function, let's see what we get tonight :-)

The txt file does not seem to update.

[RikBast]

Michel, I changed uploaded variabel to Domoticz from ‘active.power’ to ‘mppt_power’. That’s the actual output of the PV. Active.power is used power in the house, will make a different sensor for that

[MichelRabozee]

Ah, OK. So as I do not have batteries (yet, should be installed on 5-OCT-2022), I guess that is the difference between active_power and mppt_power ?

[RikBast]

I don’t think so, battery is a separate device. You can change it and see yourself

the right part of the graph is what I want to see: actual PV output and hourly as diagrams

[MichelRabozee]

OK, I will try changing the value to mppt, but so far, both values (active and mppt) are very similar: "mppt_power":0.514 "active_power":0.502

Strange I do not have the same issue of getting negative as you experience.

[MichelRabozee]

One question: you put "mppt_power" for the myCurrentW value, and you keep "total_power" for the myTotalWh value, correct ? or you changed also "total_power" with some other data ?

[RikBast]

I only changed the myCurrentW value