search

torbennehmer / hacs-e3dc

Homeassistant E3DC Integration
GNU Affero General Public License v3.0
68 stars 8 forks source link

Add energy sensors #8

Closed goebelmeier closed 1 year ago

goebelmeier commented 1 year ago

Checklist

Is your feature request related to a problem? Please describe.

Hi Torben, Congratulations on the cool first release. I am relatively close to replacing my own add-on, which is much more complex and heavyweight, with your native HA integration. What I would still miss, besides all the current power values are the energy values to use them inside the HA energy dashboard.

Describe the solution you'd like

Therefor we need at least the following sensors:

All sensors need the following attributes:

Describe alternatives you've considered

./.

Additional context

./.

torbennehmer commented 1 year ago

@goebelmeier I can do that, should not be much of a problem.

However, please be aware, that those counters have a certain variance. With my S10 I'm usually unable to really match them with the energy meter in my main fuse box and with the energy meter built into the E3DC. I haven't really found out what's going wrong in the E3DC's value, but differences up to 10 to 20 percent in the values to/from grid are quite common here. Maybe this is different with other E3DC systems, can't tell for sure.

torbennehmer commented 1 year ago

@goebelmeier I have just toyed around with those setting a bit: So far as I can tell my S10 does need a timezone adaption to work correctly. Somehow this seems broken though.

If I send him todays midnight timestamp in UTC, I get data computed starting yesterday, 22:00 local time (CEST+2). So it seems that the clock of my E3DC runs in local time, to correctly get today's data, I've to ask the S10 for 02:00 UTC = 00:00 CEST+2.

Can you confirm that with your unit and what type of unit do you use?

goebelmeier commented 1 year ago

I do have an S10 E from 2019 running S10_2023_02 with 6,5 kWh LG battery. If you have some sample code to use, i can try to reproduce that issue. Currently i'm running rscp2mqtt and i do see this phenomenon as i do have much too high energy values when restarting my add-on in the middle of the day.

AndreasTausch commented 1 year ago

Thanks first of all for the great integration. Unfortunately, except for the "Installed Battery Capacity", no other S10 sensors are displayed in the selection fields of the Energy Dashboard, although the entities are all present and also provide correct values.

So seems I have the same ambition to see the values in the energy dashboard.

Please let me have a short info what I might change in the sensor configuration.

torbennehmer commented 1 year ago

@AndreasTausch: I'm working on an update to provide appropriate sensors using the history data of the E3DC but due to some timezone weirdness, this might take a moment to get it running.

Until then, you can use the Riemann sum integral integration to create a running total over the current sensors.

torbennehmer commented 1 year ago

I do have an S10 E from 2019 running S10_2023_02 with 6,5 kWh LG battery. If you have some sample code to use, i can try to reproduce that issue. Currently i'm running rscp2mqtt and i do see this phenomenon as i do have much too high energy values when restarting my add-on in the middle of the day.

If you use the pye3dc module, you can try something like this:

from e3dc import E3DC
import json

TCP_IP = '...'
USERNAME = '...'
PASS = '...'
KEY = '...'
CONFIG = {}

e3dc = E3DC(E3DC.CONNECT_LOCAL, username=USERNAME, password=PASS, ipAddress=TCP_IP, key=KEY)

print(json.dumps(e3dc.get_db_data(), indent=2, default=str))
print(json.dumps(e3dc.get_db_data_timestamp(startTimestamp=<<insert start timestamp here>>,
    timespanSeconds=86400), indent=2, default=str))
print(json.dumps(e3dc.sendRequestTag("INFO_REQ_TIME_ZONE"), indent=2, default=str))
print(json.dumps(e3dc.sendRequestTag("INFO_REQ_UTC_TIME"), indent=2, default=str))
print(json.dumps(e3dc.sendRequestTag("INFO_REQ_TIME"), indent=2, default=str))

In the get_db_data_timestamp call, use the timestamp of midnight modified by your current timezone (e.g. +7.200 seconds for CEST) and then subtract 15 minutes = -900 seconds (weirdness in the e3dc portal), that should get you the same values as the portal for the day in question to validate things.

The other lines give the current timestamps for comparison.

torbennehmer commented 1 year ago

@goebelmeier: I've boosted my testing code a bit, could you try this one and give me the output:

Change the line tz_local=pytz.timezone("Europe/Berlin") to your local timezone if you're outside of CE(S)T, let me know what TZ you use instead. Please add the current local time your system's showing when you execute the script.

from e3dc import E3DC
import json
import pytz
from datetime import datetime, date, time

TCP_IP = 'xxx'
USERNAME = 'xxx'
PASS = 'xxx'
KEY = 'xxx'
CONFIG = {}

e3dc = E3DC(E3DC.CONNECT_LOCAL, username=USERNAME, password=PASS, ipAddress=TCP_IP, key=KEY)

e3dc_tz=e3dc.sendRequestTag("INFO_REQ_TIME_ZONE")
e3dc_local=e3dc.sendRequestTag("INFO_REQ_TIME")
e3dc_local_dt=datetime.fromtimestamp(e3dc_local)
e3dc_utc=e3dc.sendRequestTag("INFO_REQ_UTC_TIME")
e3dc_utc_dt=datetime.fromtimestamp(e3dc_utc)
delta=e3dc_local - e3dc_utc

print(f"E3DC Timezone: {e3dc_tz}, delta based on timestamps: {delta}")
print(f"e3dc-Local: {e3dc_local} = {e3dc_local_dt}")
print(f"e3dc-UTC: {e3dc_utc} = {e3dc_utc_dt}")

# E3DC timezone datetime
# May raise Unknown Timezone Error
aware_e3dc = datetime.now(pytz.timezone(e3dc_tz))
print('E3DC-TZ based local DateTime', aware_e3dc)

tz_local=pytz.timezone("Europe/Berlin")
print("tz_local", tz_local)

today = datetime.now(tz_local).replace(hour=0, minute=0, second=0, microsecond=0)

print ("Today", today)
print ("Today tuple", today.timetuple())

today_ts = int(today.timestamp())
print ("Today_ts", today_ts)

e3dc_tzinfo = pytz.timezone(e3dc_tz)
tmp = datetime.now(e3dc_tzinfo)
print("e3dc_tzinfo utcoffset", tmp.utcoffset().seconds)

today_ts += tmp.utcoffset().seconds
today_ts -= 900 # Portal Offset
today_ts -= 86400 # go to yesterday

print ("Today_ts patched", today_ts)

print(json.dumps(e3dc.get_db_data_timestamp(startTimestamp=today_ts,
    timespanSeconds=86400), indent=2, default=str))

The values you get should match the aggregated data you say for yesterday in the E3DC portal. If you run it against today (just drop today_ts -= 86400, you should get current values plus or minus some tolerance, as the portal data lags behind the live data from the unit.

Be vary, I'm not yet sure if grid_power_in/out are labeld correctly, could be that they're swapped. If they are, it would be great if you let me know that as well.

Sample:

E3DC Timezone: Europe/Berlin, delta based on timestamps: 7199.999979972839
e3dc-Local: 1686321747.000402 = 2023-06-09 16:42:27.000402
e3dc-UTC: 1686314547.000422 = 2023-06-09 14:42:27.000422
E3DC-TZ based local DateTime 2023-06-09 14:42:25.869099+02:00
tz_local Europe/Berlin
Today 2023-06-09 00:00:00+02:00
Today tuple time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=4, tm_yday=160, tm_isdst=1)
Today_ts 1686261600
e3dc_tzinfo utcoffset 7200
Today_ts patched 1686181500
{
  "autarky": 79.96076965332031,
  "bat_power_in": 4340.0,
  "bat_power_out": 4617.0,
  "consumed_production": 96.46310424804688,
  "consumption": 29572.0,
  "grid_power_in": 867.0,
  "grid_power_out": 5926.0,
  "startTimestamp": 1686181500,
  "stateOfCharge": 7.771999835968018,
  "solarProduction": 27272.0,
  "timespanSeconds": 86400
}
goebelmeier commented 1 year ago

Hi @torbennehmer, please find my output below. It was generated today at 09:23 CEST. The values match exactly yesterdays historical data from the E3/DC portal.

grid_power_in matches "NetIn" from the portal and grid_power_out matches "NetOut" from the portal.

❯ python3 e3dc-debug.py
E3DC Timezone: Europe/Berlin, delta based on timestamps: 7199.999983072281
e3dc-Local: 1686648186.000163 = 2023-06-13 11:23:06.000163
e3dc-UTC: 1686640986.00018 = 2023-06-13 09:23:06.000180
E3DC-TZ based local DateTime 2023-06-13 09:23:06.876984+02:00
tz_local Europe/Berlin
Today 2023-06-13 00:00:00+02:00
Today tuple time.struct_time(tm_year=2023, tm_mon=6, tm_mday=13, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=1, tm_yday=164, tm_isdst=1)
Today_ts 1686607200
e3dc_tzinfo utcoffset 7200
Today_ts patched 1686527100
{
  "autarky": 99.60511016845703,
  "bat_power_in": 4662.5,
  "bat_power_out": 4476.0,
  "consumed_production": 29.461923599243164,
  "consumption": 15194.0,
  "grid_power_in": 36234.0,
  "grid_power_out": 60.0,
  "startTimestamp": 1686527100,
  "stateOfCharge": 64.28600311279297,
  "solarProduction": 54308.0,
  "timespanSeconds": 86400
}

afterwards i've commented out that line today_ts -= 86400 # go to yesterday and the result was matching todays values from the portal:

❯ python3 e3dc-debug.py
E3DC Timezone: Europe/Berlin, delta based on timestamps: 7199.999982118607
e3dc-Local: 1686648467.000302 = 2023-06-13 11:27:47.000302
e3dc-UTC: 1686641267.00032 = 2023-06-13 09:27:47.000320
E3DC-TZ based local DateTime 2023-06-13 09:27:46.586173+02:00
tz_local Europe/Berlin
Today 2023-06-13 00:00:00+02:00
Today tuple time.struct_time(tm_year=2023, tm_mon=6, tm_mday=13, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=1, tm_yday=164, tm_isdst=1)
Today_ts 1686607200
e3dc_tzinfo utcoffset 7200
Today_ts patched 1686613500
{
  "autarky": 98.97632598876953,
  "bat_power_in": 587.0,
  "bat_power_out": 2416.0,
  "consumed_production": 98.97632598876953,
  "consumption": 3126.0,
  "grid_power_in": 32.0,
  "grid_power_out": 32.0,
  "startTimestamp": 1686613500,
  "stateOfCharge": 31.190000534057617,
  "solarProduction": 1600.0,
  "timespanSeconds": 86400
}
torbennehmer commented 1 year ago

@goebelmeier Thanks for the test run, that matches what I'm seeing. I'm currently building some first implementation, stay tuned.

bf8392 commented 1 year ago

@AndreasTausch: I'm working on an update to provide appropriate sensors using the history data of the E3DC but due to some timezone weirdness, this might take a moment to get it running.

Until then, you can use the Riemann sum integral integration to create a running total over the current sensors.

Can you show how to configure the Riemann helper? Maybe it would be enough to mention this in docs...because I think it gives nice mulitple values this way...

torbennehmer commented 1 year ago

Current Git HEAD has a first implementation of the integrated sensors. I'll get a release out in a few days, as soon as I cleanup a few more things, the sensor names are highly inconsistent right now, so I'd advise to wait for the next release for productive usage.

torbennehmer commented 1 year ago

@bf8392 As for the Riemann Integration, it looks quite simple, before, with my ModBus integration, it looked something like this:

- platform: integration
  unique_id: e3dc_netzbezug_menge
  name: "E3DC: Netzbezug Menge"
  source: sensor.e3dc_netzbezug_leistung
  unit_prefix: k
  round: 2
  method: left

- platform: integration
  unique_id: e3dc_netzeinspeisung_menge
  name: "E3DC: Netzeinspeisung Menge"
  source: sensor.e3dc_netzeinspeisung_leistung
  unit_prefix: k
  round: 2
  method: left

- platform: integration
  unique_id: e3dc_pv_menge
  name: "E3DC: PV Menge"
  source: sensor.e3dc_pv_leistung
  unit_prefix: k
  round: 2
  method: left

etc.

bf8392 commented 1 year ago

Wow great great thanks! That's absolutely Amazing!!! :-)

torbennehmer commented 1 year ago

@goebelmeier @bf8392 Just one more info: From my point of view, the E3DC DB values are with a significant tolerance, I've seen 10% more than once when comparing to my Tibber Smart Meter reader. I am currently not sure who or what is correct here, so this will stay a bit of a guessing game for now. I'll also have to compare this to the additional energy meter built into my E3DC I use for my taxes. I'll keep an eye on it, but I can't tell how much time it'll take to really get an handle on this.

torbennehmer commented 1 year ago

Here you go, v2.0.0 will help you:

image

Please test, I'd appreciate any feedback. If you experience problems, please open a new bug.

AndreasTausch commented 5 months ago

Dear Torben, Still using the version 1.0.0. of your great Integration. Was always Ranking bery well.

What about the sequence to update? Only download the newest Release via HACS and the reboot?

Or better deinstall, Download and New Installation?

Thanks for your advice!

Andreas

14.06.2023 20:37:55 torbennehmer @.***>:

@goebelmeier[https://github.com/goebelmeier] Thanks for the test run, that matches what I'm seeing. I'm currently building some first implementation, stay tuned.

— Reply to this email directly, view it on GitHub[https://github.com/torbennehmer/hacs-e3dc/issues/8#issuecomment-1591795508], or unsubscribe[https://github.com/notifications/unsubscribe-auth/A7Q2LEMOWKAKJEJP3UJAY53XLIAH7ANCNFSM6AAAAAAY2QFMDI]. You are receiving this because you were mentioned. [Verfolgungsbild][https://github.com/notifications/beacon/A7Q2LEN7BSJPCKTPMROXPDLXLIAH7A5CNFSM6AAAAAAY2QFMDKWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTS64DPTI.gif]