Closed roninf closed 1 year ago
I might be wrong but some things with the data shown in the energy view does not seem to match with what is shown by the solarlog itself. I think it might be related to how the sensors are used....
I assume the solar production (yield total) sensor does work?. Solarlog does not have a return to grid sensor afaik.
So, you should only be able to use
Ah, I think I get your point. I don’t have a consumption meter connected myself, so hard to check for me. Let’s first see what is being send by the sunwatcher pypi package and by the device
solarlog consumption total is according to the API documentation “ Accumulated total consumption, all Consumption meters” in Wh (send by the device)
solarlog yield total is according to the API documentation “Total yield from all of the inverters” in Wh (send by the device).
solarlog_power_available = power AC - consumption AC in Watt (Calculated by the pypi package)
power AC is according to the API documentation “ Total output PAC from all of the inverters and meters in inverter mode” in Watt (send by the device)
consumption AC is according to the API documentation “ Current total consumption PAC from all of the consumption meters” in Watt (send by the device)
These Watt measurements are not intended for the energy view, you need a (k)Wh measurement. So, we only can use the first two.
question. Is the “consumption total” an increasing value, or is it also going down when returning to grid?
solarlog documentation solarlog source (message by IssueLinks)
For me solarlog_power_available is the correct sensor, if the value is negative you are using this amount of power from the grid if the value is positive this is the amount of power returned to grid. It seems to be different to other integrations unfortunately. consumption total is the value of the energy consumption since the device/stats have been reset. So its an increasing value. sensor.solarlog_consumption_ac seems to be the same as power available. (but only positive values)
But power_available, consumption_AC and power_AC are all measurements in Watt, not in kWh. The energy view needs measurements in kWh.
See also page 298 of the manual https://www.solar-log.com/manuals/manuals/en_GB/SolarLog_Manual_4x_EN.pdf
Power_ac = index 101 —> W Consumption_ac = index 110 —> W Power available = power_ac - consumption_ac —> so also W
Btw, you can use template sensors to create two new sensors, one with only the positive values, and one with only the negative values (multiply these with -1). But again, you should use a kWh sensor (consumption total).
the current state_class of the “ consumption total” sensor is “total_increasing”, but we might need to change this to “measurement” as the connected consumption meter can be a two way meter, so can also go down.
So, for now, try to overwrite the state_class of the “consumption total” sensor to “measuring”. You can do this in the customize menu in HA.
I know i am just hoping that it can integrated somehow... because this energy thing is kind of genius:)
If there is anything i should test or provide you info on let me know as i have the following device in use here:
Yes, sure, if there is a general solution that works for everyone, we can add it to the integration. But it looks like there are multiple ways to install the consumption meter(s), at least that is what they say in the manual, so I’m not convinced there is a general solution.
But we first need to agree on the fact that we need a kWh meter, do you agree?
I guess you are right :) What is kind of dumb is the Solarlog integrated webserver of the SolarlogBase provides all these data, as they are shown within webinterface. So i am wondering hwo we can "extract" these cause they are definetly there...
Yes, but I think they decided to only put limited information in the json output (see page 298 of the manual). So the solarlog has way more info, but they decided to not send all info in the json output unfortunately.
But lets try what we can make of the info we have.
We have the Total solar energy generation of the panels (yield total) and we have the total of all consumption meters that you have connected. I'm not sure what info it gives in your case, as I'm not sure which device is measuring what. Is it possible to find out what is corresponding the the "consumption total"? I would expect the energy that is going in or out from the grid (electrical company), but I'm not sure. You might be able to check with the meter from the energy firm you buy your electricity from.
But, did you try changing the "state_class" to measurement? This might solve it already?
So consumption total as i understand is the total in kwh that is used independent from what is solar energy or grid usage. So in my case thats the energy my whole house is using since the device was last reset. This data is measured by the solarlog meter not the energy company meter. So in my case its 4094.85 right now and increasing with every kw that is used by the whole house. Total yield is what is produced by the solar panels in my case 10777.65 and increasing.
How and where can i change the sensor class? I guess these are given by the integration ?
Yes, I think so yes, but it all depends on where the energy meter is installed. In the solarlog manual I see two options (see section 25.3), an option
I do not entirely understand the difference, perhaps the first is only recoding pulses (always positive), while the second can also go down, not sure.
Anyway. the state_class can be changed via "configuration" --> customisations --> select the entity --> click on the pencil of state_class and change to measurement
Hm ok i am not using the same solarlog but the it should be the same thing anyway. In my case the solarlog meter is place directly before the energy companies meter. Its working 2 way so i think you have one value for kwh used from grid and one value for what is returned to the grid. In the end you could also count the pulses comeing from the leds. 25.3.2 is kind of the its installed in my house at least in regards to both meters before the grid
I found this info in the docs
I would first try to only use the grid consumption = "solarlog consumption total" (with state_class changed to "measurement").
If that doesn't work, try making the two template sensors as described before). It might need some more enhanced formula, like "calculate the difference between the the current and previous measurement. If negative, add it to template sensor "grid in", if positive, add in to template sensor "grid out".
I have changed sensor.solarlog_consumption_total to measurement now and see what happens.
Also keep an eye on #55405, where we discuss how we should deal with Net energy measurements (Net = grid in - grid out). I think sensor.solarlog_consumption_total is the Net energy (at least, that is what the solarlog documentation says).
Ok so changing the state_class does not matter it does not change anything. At least nothing i would have recognized. Next i will try to create the template sensors. Do you happen to know how to do that ?
I have just talked to the solarlog support chat, which was kind of what i expected. You cant get the data as shown in the webserver. I don´t how this integration is working but i guess you are using the open JSON to get the data right? In the handbook there is a list of what we can get using json. Maybe we can achieve what we need using that ?
Yes, it is using the open JSON info.
The only relevant info (for the energy view) in the open JSON info is the solarlog total yield (production) of the solar panels and the total of the consumption meters.
I'm don't think with these two measurements, we are able to calculate the individual values.
This page explains the needed parameters, we need. In between brackets the ones we have in the JSON output.
or
I have the first option myself, solar_energy from the solarlog and the two grid entities from the smart meter (DSMR). But with the current output from the solarlog alone, you won't be able to get all the required info. I think you should make a request to solarlog to ask them to implement a split up (the info is in there).
First of all thanks to all contributors for the great work so far!
I just got my JSON-interface activated this week and tried out the solar-log integration and apparently experienced the above mentioned limits. It seems that solar_yield and consumation are added up, however no return to grid is considered (I cannot choose a respective entity in the preferences). As a result, I have a huge consumption in the last two days: Just for background, I have a two-way meter installed, i.e. solar production is deducted at consumption meter.
In this blog (only available in German) I read that it should be possible to get information about additional inverters (e.g. heatpump) with the following payload: {"782":null}
. Unfortunately this did not work for me. I contacted the author but have not yet received any feedback.
With regards to the different installation methods, would it be possible to catch this with a respective property that can be set in the configuration?
Furthermore, for a set-up with 2-way meter (and no battery) the following could apply (not yet fully thought throug): If delta solar_energy is positive and delta grid_energy is negative, grid_energy is amount returned to grid and difference is consumption (home energy). If delta solar_energy is positive and delta grid_energy is positive, difference is consumption (home energy). If no solar_energy, delta grid_energy is consumption. I'm not sure if with the above the missing information could be calculated in solar-log integration or if this would have to be done in HA itself.
@dontinelli There can be two ways to add this, in the Solarlog integration itself or in HA by using template sensors. I propose you first figure out if it can be achieved with template sensors. When you have figured out the logic and after we have checked that it worked, we can always add the logic in the Solarlog integration.
Here is an example of a template sensor I use myself (combination of solarlog and a DSMR sensor, but you get the idea).
template:
- sensor:
- name: "Net energy use"
unique_id: "energy_net"
unit_of_measurement: kWh
state_class: "measurement"
device_class: "energy"
state: "{{ (states('sensor.solar_yield_day') | float - states('sensor.dsmr_day_consumption_electricity_returned_merged') | float + states('sensor.dsmr_day_consumption_electricity_merged') | float) | round(2) }}"
- sensor:
- name: "Energyuse home"
unique_id: "energy_home_usage"
unit_of_measurement: kWh
state_class: "measurement"
device_class: "energy"
state: "{{ (states('sensor.solar_yield_day') | float - states('sensor.dsmr_day_consumption_electricity_returned_merged') | float) | round(2) }}"
Note, apparently the format of template sensors is slightly different, see https://www.home-assistant.io/integrations/template/#configuration-variables, so without testing (no time now), it should be something like this. I will check the correct format tomorrow. You might need to add the last_reset attribute, but that does not seem to be documented.
Will somehow have to try that :) I should have gone with SMA as my Inverter is SMA Tripower anyway. Well i guess will might figure out a way to get that done. If not i need to count the pulses on the meter i guess ;)
@dontinelli please let me know if you get that templates running!
Updated the templates to include state_class and device_class. Cannot test it right now, but will check the format tomorrow.
A small break-through, not on the template but on the consumption. When deactivating the password protection for the user in the solar-log, the above REST request with payload {"782":null}
returns the consumption (in W) of the different inverters (solar panel, total consuption, heatpump).
I also have the current consumption in W as well as the power available (negative and positive values) also in W (both provided throught the actual integration without issues) but that is not usable for the energy "page" as it needs to be kwh or kw if understood correctly.
I also have the current consumption in W as well as the power available (negative and positive values) also in W (both provided throught the actual integration without issues) but that is not usable for the energy "page" as it needs to be kwh or kw if understood correctly.
For the conversion from W to kWh the following could be helpful: integration
With this I can integrate the heatpump into the energy dashboard:
sensor:
- platform: rest
resource: http://xxx/getjp
method: POST
payload: '{"782":null}'
value_template: '{{ value_json["782"]["2"] }}'
name: power_heatpump_w
unit_of_measurement: W
device_class: power
scan_interval: 15
force_update: true
- platform: integration
source: sensor.power_heatpump_w
name: energy_heatpump_integration
unit_prefix: k
round: 2
template:
- sensor:
- name: "Verbrauch Wärmepumpe"
unique_id: "energy_heatpump"
unit_of_measurement: kWh
state_class: "measurement"
device_class: "energy"
state: "{{ states('sensor.energy_heatpump_integration') | float }}"
I think you should use state_class: “total_increasing”
if the measurement is always increasing.
ok got this now:
platform: integration source: sensor.solarlog_power_available name: energy_spent unit_prefix: k round: 2
and the sensor shows a negative value of 0.13 and increasing but to be honest for now i dont exactly know what to do with that :) I guess i somehow have to get two sensors: if positive = send to grid if negative = consumed from grid (with a positive value, for the energy page)
Power available = AC power from solar panels - AC consumption (both in Watt) (see https://bitbucket.org/Lavode/sunwatcher/src/master/sunwatcher/solarlog/solarlog.py)
This will result in the same info you already have.
(at least, in theory)
There seems to be more data accessible, see https://github.com/iobroker-community-adapters/ioBroker.solarlog/blob/master/main.js
documented better here https://github.com/iobroker-community-adapters/ioBroker.solarlog/blob/master/docs/solarlog_dataobjects.md
Promising data: {"878":null} yearly values for production (Wh), consuption (Wh) and self-consumption (kWh).
There seems to be more data accessible, see https://github.com/iobroker-community-adapters/ioBroker.solarlog/blob/master/main.js
documented better here https://github.com/iobroker-community-adapters/ioBroker.solarlog/blob/master/docs/solarlog_dataobjects.md
Promising data: {"878":null} yearly values for production (Wh), consuption (Wh) and self-consumption (kWh).
Indeed very interesting. I guess production ./. self-consumption would then result in the energy returned to the grid. I have now implemented the calculation based on the power surplus (with integration). Curious to see tomorrow if this functions as expected.
I still are not able to calculate all info. We have the three numbers with the green arrow, but you would need one of the red arrows as well, I guess
Hm ok thats exactly what we somehow need to calculate i guess. What we have is grid and out in W (depending on the value +/-) maybe we can manage that with the energy integration thing?
Yes, I was just hoping this info is directly available, in stead of integrating. And I remembered from Math class that with three equation and three unknowns, you should be able to solve the unknowns. But I tried and didn’t succeed,
I still are not able to calculate all info. We have the three numbers with the green arrow, but you would need one of the red arrows as well, I guess
Is the "self consumption" provided by solar log really the total consumption of the household or rather the self-consumed solar power?
I don't know, I don't have any production sensors connected myself. So I was hoping you could tell me 😄 Perhaps you can compare the numbers with what you can see in the Solarlog.
I compared the JSON output with my dashboard from the energy company. I could not fully reconcile all amounts. It seems, though, that consumption is not the grid consumption but the total consumption of the household (i.e. grid and self-consumed solar production). Production is clear. Self consumption I could not yet check, as I struggle with the JSON return:
{"878":[["01.01.20",7115585,3584433,1639,0,0,0],["01.01.21",9697742,5017369,2401,0,0,0]]}
It seems that the 878-payload returns the data for last and this year (production, consumption, self-consumption). How can I now get these values (of the current year) into the sensor (sorry, I'm rather new to HA).
Self consumption is within the solarlog webserver is the amount engery used from solar produced energy and not the consumption of the house. total consumption is what reflects the total house consumption at least i think so. Problem is you cant get that out of the solarlog as far as i know.
After some hours of coding, testing and adapting of the statistics database (I had to manually delete some old wrong values) I came to the following set-up, which seems to work:
sensor:
- platform: rest
resource: http://xxx:8002/getjp
method: POST
payload: '{"854":null}'
value_template: '{{ (value_json["854"].1.1.1 / 1000) | float}}'
name: energy_consumption_year
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx:8002/getjp
method: POST
payload: '{"854":null}'
value_template: '{{ (value_json["854"].1.1.2 / 1000) | float}}'
name: energy_heatpump_year
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx:8002/getjp
method: POST
payload: '{"878":null}'
value_template: '{{ (value_json["878"].1.1 / 1000) | float}}'
name: energy_production
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx:8002/getjp
method: POST
payload: '{"878":null}'
value_template: '{{ (value_json["878"].1.2 / 1000) | float}}'
name: energy_consumption
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx:8002/getjp
method: POST
payload: '{"878":null}'
value_template: '{{ value_json["878"].1.3 | float}}'
name: energy_self_consumption
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
template:
- sensor:
- name: "energy_return"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: >
{% if ((states('sensor.energy_production') | float) > 0 and (states('sensor.energy_self_consumption') | float) > 0) %}
{{(states('sensor.energy_production') | float) - (states('sensor.energy_self_consumption') | float) }}
{% endif %}
- name: "energy_consumption_grid"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: >
{% if ((states('sensor.energy_consumption') | float) > 0 and (states('sensor.energy_self_consumption') | float) > 0) %}
{{ (states('sensor.energy_consumption') | float) - (states('sensor.energy_self_consumption') | float) }}
{% endif %}
- name: "energy_consumption_heatpump"
unique_id: "energy_consumption_heatpump"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: "{{ states('sensor.energy_heatpump_year') | float }}"
- name: "energy_consumption_household"
unique_id: "energy_consumption_household"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: "{{ (states('sensor.energy_consumption_year') | float) - (states('sensor.energy_heatpump_year') | float) }}"
I used the entities as follows in the energy dashboard:
Of course I will have to keep an eye on the development and compare with the direct solar-log output and let you know.
As said on an earlier occasion, the additional payloads only are accessible if the authentication is disabled. After I verified the appropriateness of the values, I will check if the Rest request also works with enabling authentication and using credentials.
One problem I experienced is that the (daily?) restart of the solar-log leads to wrong values in the calculated entities. I tried to catch this with the if-clause in the templates. I know this is only a quick and dirty workaround, but one thing at the time. Feel free to comment on my code, happy to implement any improvement.
One problem I experienced is that the (daily?) restart of the solar-log leads to wrong values in the calculated entities.
You might want to try the month or year values, in stead of the day totals. These might work better with the "sensor_class" "total_increasing". Every times the measurement goes back to zero, it is "resetting". The energy view should be able to handle this, but with small values, it can be tricky and lead to errors. Better to use long term data (like year data, or total data). See also https://developers.home-assistant.io/blog/2021/08/16/state_class_total/
You're right. As you can see in my code, I calculate based on total (or year) consumption. The problem is, that the REST request returns an invalid amount, if solar-log is restarting. And when you calculate based on these values for the tempalte, unreasonable results can come out. This totally spoiled the statistic.
Yes, that is right, in the few minutes after a restart of the solarlog (which happens every night in my case), it will return zero values. In the solarlog integration, I have filtered these out, by looking at the timestamp (it sends 1-1-1999 in these "false" messages) and ignore these messages.
if data.time.year == 1999:
raise update_coordinator.UpdateFailed(
"Invalid data returned (can happen after Solarlog restart)."
)
You guys are awesome.. I'm silently following this thread since I have the same issue :)
@dontinelli I'm working on getting the additional info in the integration. To do that, I first need to modify the "sunwatcher" pypi package, that we use. I asked for the 854
data in the same way as the 801
data, but I get {'854': 'ACCESS DENIED'}
. I know your solution is to deactivate the user password, but did you had any luck with the password still activated?
@Ernst79 : not yet. But I did not try too long, as I did not want to jeopardize my statistics again. As my solar-log is behind a seperate router where I can limit the access, for me the risk is bearable for the testing period.
Here an update from my tests.
To catch errors in the calculation, I updated the configuration slightly. For the sake of completeness here the current config:
sensor:
- platform: rest
resource: http://xxx/getjp
method: POST
payload: '{"854":null}'
value_template: '{{ (value_json["854"].1.1.1 / 1000) | float}}'
name: energy_consumption_year
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx/getjp
method: POST
payload: '{"854":null}'
value_template: '{{ (value_json["854"].1.1.2 / 1000) | float}}'
name: energy_heatpump_year
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx/getjp
method: POST
payload: '{"878":null}'
value_template: '{{ (value_json["878"].1.1 / 1000) | float}}'
name: energy_production
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx/getjp
method: POST
payload: '{"878":null}'
value_template: '{{ (value_json["878"].1.2 / 1000) | float}}'
name: energy_consumption
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
- platform: rest
resource: http://xxx/getjp
method: POST
payload: '{"878":null}'
value_template: '{{ value_json["878"].1.3 | float}}'
name: energy_self_consumption
unit_of_measurement: kWh
device_class: energy
scan_interval: 60
force_update: true
template:
- sensor:
- name: "energy_return"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: >
{% if ((states('sensor.energy_production') | float) > 0 and (states('sensor.energy_self_consumption') | float) > 0) %}
{% if ((states('sensor.energy_production') | float) - (states('sensor.energy_self_consumption') | float)) > (states('sensor.energy_return') | float) %}
{{(states('sensor.energy_production') | float) - (states('sensor.energy_self_consumption') | float) }}
{% else %}
{{states('sensor.energy_return')}}
{% endif %}
{% else %}
{{states('sensor.energy_return')}}
{% endif %}
- name: "energy_consumption_grid"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: >
{% if ((states('sensor.energy_consumption') | float) > 0 and (states('sensor.energy_self_consumption') | float) > 0) %}
{% if ((states('sensor.energy_consumption') | float) - (states('sensor.energy_self_consumption') | float)) > (states('sensor.energy_consumption_grid') | float) %}
{{ (states('sensor.energy_consumption') | float) - (states('sensor.energy_self_consumption') | float) }}
{% else %}
{{states('sensor.energy_consumption_grid')}}
{% endif %}
{% else %}
{{states('sensor.energy_consumption_grid')}}
{% endif %}
- name: "energy_consumption_heatpump"
unique_id: "energy_consumption_heatpump"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: >
{% if (states('sensor.energy_heatpump_year') | float) > 0 %}
{{ states('sensor.energy_heatpump_year') | float }}
{% else %}
{{states('sensor.energy_consumption_heatpump')}}
{% endif %}
- name: "energy_consumption_household"
unique_id: "energy_consumption_household"
unit_of_measurement: kWh
state_class: "total_increasing"
device_class: "energy"
state: >
{% if ((states('sensor.energy_consumption_year') | float) > 0 and (states('sensor.energy_heatpump_year') | float) > 0) %}
{% if ((states('sensor.energy_consumption_year') | float) - (states('sensor.energy_heatpump_year') | float)) > (states('sensor.energy_consumption_household') | float) %}
{{ (states('sensor.energy_consumption_year') | float) - (states('sensor.energy_heatpump_year') | float) }}
{% else %}
{{states('sensor.energy_consumption_household')}}
{% endif %}
{% else %}
{{states('sensor.energy_consumption_household')}}
{% endif %}
Basically the above configuration works with the energy board and populates the dashboard:
I also compared the output with the Solar-Log dashboard. Based on this I think the current configuration is correct. However, I encountered some differences in the "return to grid", "consumption from grid" and "total consumption". I think the difference in the first two values is due to the fact that those are calculated based on the self-consumption, which solar-log only provides in kWh instead of Wh. The total consumption is then calculated by HA to match with the other values provided.
The "energy consumption household" and "energy consumption heatpump" are quite accurate, I only encountered minor rounding differences I have not yet further investigated.
Would be good if someone else could confirm (or correct) the finding. Furthermore, as a next step we should find out if it is possible to access the additional information also with credentials activated. @roninf : perhaps you could support as you have different solar-log devices in operation. This should also be possible outside of HA with curl:
curl -X POST -H "Content-Type: application/json" -d '{"782":null}' -u 'username' [IP]/getjp
As far as I understand, the username is always "user", the password is as you have specified in the solar-log settings.
The problem
The solarlog integration added to the energy management within the latest beta is not working as expected. You can only choose "solarlog consumption total" for grid usage and return to grid which is not working because solarlog provides a sensor that is called sensor.solarlog_power_available. This sensor is showing a negative value if power is drawn from the grid and a positive value if power is returned to grid.
What is version of Home Assistant Core has the issue?
core-2021.9.0b1
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
solarlog
Link to integration documentation on our website
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
No response
Additional information
No response