search

suaveolent / ha-hoymiles-wifi

Home Assistant custom component for Hoymiles DTUs and the HMS-XXXXW-2T microinverters
MIT License
129 stars 9 forks source link

Wrong readings when cloud is blocked #25

Closed Julian0021 closed 4 months ago

Julian0021 commented 6 months ago

When the DTU has no access to the internet, every reading in an interval of exactly 3 minutes is 0. All readings in between are correct.

image

Julian0021 commented 6 months ago

This does not affect actual production, just the reading is wrong.

suaveolent commented 6 months ago

That it interesting and reminds me of #11.

I assume you are already running the latest version?

Julian0021 commented 6 months ago

Yes, running latest release available in HACS (and latest dtu firmware)

suaveolent commented 6 months ago

If you enable cloud access, do the values normalize?

Also to ensure: Are you running only the custom component or anything else in parallel which accesses the inverter?

suaveolent commented 6 months ago

I know @dicer is running the inverter without Internet access successfully. Maybe he can help?

dicer commented 6 months ago

I'm still running the factory firmware cause update doesn't work as planned offline. I don't see your pattern though. The old firmware version is regularly dropping from the wifi and I get very sporadic readings only (same on two inverters).

What is HACS btw?

Julian0021 commented 6 months ago

If you enable cloud access, do the values normalize?

Also to ensure: Are you running only the custom component or anything else in parallel which accesses the inverter?

Yes, as soon as cloud access is reenabled the values normalize. And no, this integration is the only thing messing with the inverter

suaveolent commented 6 months ago

@Julian0021 I have been trying to reproduce this issue. However, my inverter does not show the same behaviour.

Are you running the latest firmware?

Should be:

dtu_hw_version: "H00.01.00"
dtu_sw_version: "V00.01.11"
inverter_hw_version: "H00.04.00"
inverter_sw_version: "V01.00.08"

Maybe it has to do with the way you block Internet access?

@dicer HACS is the home assistant community store you can use to install this component: https://hacs.xyz

Julian0021 commented 6 months ago

@suaveolent

I am running dtu SW version V01.01.09, otherwise its the same

So maybe this issue didnt exist on older firmware?

I block internet out using a traffic rule on my unifi gateway which applies to the vlan the dtu is in.

suaveolent commented 6 months ago

@suaveolent

I am running dtu SW version V01.01.09, otherwise its the same

So maybe this issue didnt exist on older firmware?

I block internet out using a traffic rule on my unifi gateway which applies to the vlan the dtu is in.

Which device are you using? this is not a HMS-XXXW-2T device right? Are you using DTU-Wlite or similar?

Julian0021 commented 6 months ago

Which device are you using? this is not a HMS-XXXW-2T device right? Are you using DTU-Wlite or similar?

I am running the HMS-800W-2T. No DTU other than the integrated one. Maybe this version isnt yet available for your region? I live in Germany.

suaveolent commented 6 months ago

Which device are you using? this is not a HMS-XXXW-2T device right? Are you using DTU-Wlite or similar?

I am running the HMS-800W-2T. No DTU other than the integrated one. Maybe this version isnt yet available for your region? I live in Germany.

That is weird indeed. Especially, since the previous firmware versions started with V00. Did you perform an upgrade via hoymiles cloud or did the inverter ship with this version?

Julian0021 commented 6 months ago

Which device are you using? this is not a HMS-XXXW-2T device right? Are you using DTU-Wlite or similar?

I am running the HMS-800W-2T. No DTU other than the integrated one. Maybe this version isnt yet available for your region? I live in Germany.

That is weird indeed. Especially, since the previous firmware versions started with V00. Did you perform an upgrade via hoymiles cloud or did the inverter ship with this version?

Sorry. It seems I have mixed something up. My versions are:

Inverter SW Version V01.01.09 Inverter HW Version H00.04.00

DTU SW Version V00.01.11 DTU HW Version H00.01.00

Julian0021 commented 6 months ago

I was prompted to update the device upon first installation, which I did (4 days ago)

suaveolent commented 6 months ago

Got it. So only the inverter firmware is different.

Can you maybe try something different: instead of blocking the inverter via firewall rules, can you just unplug your Internet connection and check if the issue still persists?

Julian0021 commented 6 months ago

Got it.

So only the inverter firmware is different.

Can you maybe try something different: instead of blocking the inverter via firewall rules, can you just unplug your Internet connection and check if the issue still persists?

I just tested this, exactly the same behavior

suaveolent commented 6 months ago

Got it. So only the inverter firmware is different. Can you maybe try something different: instead of blocking the inverter via firewall rules, can you just unplug your Internet connection and check if the issue still persists?

I just tested this, exactly the same behavior

Hm. What update interval are you using?

Julian0021 commented 6 months ago

The default value of 35

Julian0021 commented 6 months ago

All sensors (including dtu) report the same value they do at night for some readings when its disconnected from the internet

suaveolent commented 6 months ago

That is some weird issue you have.

I have blocked my inverter from Internet access since yesterday morning and still cannot reproduce the issue. Only difference I can see so far is your inverter SW version. Maybe hoymiles introduced sth. in the new SW version which triggers this effect.

You can try updating to the latest v0.2.0 version and see if there is an improvement (although there should not really be one).

I will see if I can implement a workaround in one of the upcoming versions to mitigate this issue. The problem, however, seems to come from the DTU and not the custom component.

Julian0021 commented 6 months ago

That is some weird issue you have.

I have blocked my inverter from Internet access since yesterday morning and still cannot reproduce the issue.

Only difference I can see so far is your inverter SW version. Maybe hoymiles introduced sth. in the new SW version which triggers this effect.

You can try updating to the latest v0.2.0 version and see if there is an improvement (although there should not really be one).

I will see if I can implement a workaround in one of the upcoming versions to mitigate this issue. The problem, however, seems to come from the DTU and not the custom component.

No question about it. The dtu seems to behave weirdly when disconnected from the cloud. Not really a bug from the integration, but I was hoping you could mitigate it.

suaveolent commented 6 months ago

Noticed today that since yesterday I was indeed experiencing the same issue. I will keep you updated if I find a solution for it

O-misses commented 5 months ago

I've got a very similar behavior: HMS-800W-2T dtu_hw_version: "H00.01.00" dtu_sw_version: "V00.01.11" inverter_hw_version: "H00.04.00" inverter_sw_version: "V01.00.08" 60s update intervall. Wifi signal strength 58% Internet access blocked via firewall (Fritz Box router) Connection loss is somewhat regular (see graph), but not every 3min. Actual energy delivery is not affected.

I guess it is a problem within the DTU firmware, and has nothing to do with this HA integration. When I use the S-miles installer app to access the inverter over its own wifi, the displayed values often freeze after some time. Although there are clouds in front of the sun, the values stay the same. Maybe the DTU is simply unresponsive whilst trying to send data to the cloud?

Graph
suaveolent commented 5 months ago

I've got a very similar behavior: HMS-800W-2T dtu_hw_version: "H00.01.00" dtu_sw_version: "V00.01.11" inverter_hw_version: "H00.04.00" inverter_sw_version: "V01.00.08" 60s update intervall. Wifi signal strength 58% Internet access blocked via firewall (Fritz Box router) Connection loss is somewhat regular (see graph), but not every 3min. Actual energy delivery is not affected.

You graph seems to make sense since you are using a 60s polling interval as opposed to 35 seconds, which would explain the larger gaps. If I could find out a reliable way whether the DTU is unresponsive, I could maybe implement a workaround.

IIChrisII commented 5 months ago

HMS-800W-2T dtu_hw_version: "H00.01.00" dtu_sw_version: "V00.01.11" inverter_hw_version: "H00.04.00" inverter_sw_version: "V01.00.08" update interval: 35 (default)

I have exactly the same disconnections. My assumption is that the dtu is trying to open the socket to colud ip and this blocks the complete dtu until socket connection timeout. To prove that we would need a fake machine at that cloud ip inside virtual lan sending back connection refused. But I think building up that scenario is to much effort for a simple test.

However @suaveolent could you add a delay for setting entities unavailable? I mean if the dtu does not response, could you keep all values of the entities and set them finally unavailable after dtu is not responding for more than 70 seconds (two default cycles should be enough)?

suaveolent commented 5 months ago

HMS-800W-2T dtu_hw_version: "H00.01.00" dtu_sw_version: "V00.01.11" inverter_hw_version: "H00.04.00" inverter_sw_version: "V01.00.08" update interval: 35 (default)

I have exactly the same disconnections. My assumption is that the dtu is trying to open the socket to colud ip and this blocks the complete dtu until socket connection timeout. To prove that we would need a fake machine at that cloud ip inside virtual lan sending back connection refused. But I think building up that scenario is to much effort for a simple test.

However @suaveolent could you add a delay for setting entities unavailable? I mean if the dtu does not response, could you keep all values of the entities and set them finally unavailable after dtu is not responding for more than 70 seconds (two default cycles should be enough)?

Yes, that might be a good idea. I will look into this.

DirkBaumeister commented 5 months ago

Hi, any update on this topic? :) I am currently deciding which inverter I wanna get and this behaviour when the internet is blocked would be not so great.

Thanks!

suaveolent commented 5 months ago

Hi, any update on this topic? :) I am currently deciding which inverter I wanna get and this behaviour when the internet is blocked would be not so great.

Thanks!

I might be able to implement a workaround which only sets the data to 0 after a certain delay. However, since the inverter does not respond, the displayed data will also be not accurate and only display the data from the last response.

So in the end, Hoymiles needs to fix this. Maybe you can file a bug report and hope for a firmware update?

DirkBaumeister commented 5 months ago

So I guess it's the best for now to drop the request to the hoymiles cloud entirely to prevent this lockup I guess :) Unfortunatly I don't have this inverter to test it. Anyone that has the ability to test this with a firewall or something? :)

N3rdix commented 5 months ago

I will block access for my HMS-800W-2T for the next couple of hours and report back. If no-one else is faster I can also report an issue to Hoymiles then 😄

DirkBaumeister commented 5 months ago

I will block access for my HMS-800W-2T for the next couple of hours and report back. If no-one else is faster I can also report an issue to Hoymiles then 😄

Thank you 😄

N3rdix commented 5 months ago

Well, i am able to reproduce the issue but wondering at the same time how we could convince Hoymiles that we have issues when accessing their devices via an unreleased API 😕 Although it seems strange that those zero values only come in a 3-minute interval that's what I observed when logging the outgoing requests from the DTU. Snag_34036af2

1. test with all outgoing traffic blocked. I observed the device overwrites my local DNS settings as it seems and set 8.8.8.8 and 1.1.1.1 as fixed values:

"2024-04-29 09:59:05","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3989 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:59:02","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3988 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:58:59","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3987 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:58:56","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3984 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:58:53","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3983 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:58:50","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3980 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:58:47","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3979 PROTO=TCP SPT=62199 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:56:05","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3940 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:56:02","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3939 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:55:59","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3938 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:55:56","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3937 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:55:53","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3934 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:55:50","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3931 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:55:47","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:07:00:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=3930 PROTO=TCP SPT=62198 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 09:53:00","warning","kern","kernel","ubnt"," IPv4: martian destination 0.0.0.0 from 10.10.30.73, dev eth1.30"
"2024-04-29 09:52:57","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=8.8.8.8 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3885 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:55","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=8.8.8.8 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3882 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:54","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=8.8.8.8 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3881 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:53","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=8.8.8.8 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3880 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:50","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=1.1.1.1 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3877 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:48","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=1.1.1.1 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3876 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:47","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=1.1.1.1 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3875 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:52:47","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:00:00:41 SRC=10.10.30.73 DST=1.1.1.1 LEN=65 TOS=0x00 PREC=0x00 TTL=63 ID=3874 PROTO=UDP SPT=26339 DPT=53 LEN=45 MARK=0x65000000 "
"2024-04-29 09:43:24","warning","kern","kernel","ubnt"," IPv4: martian destination 0.0.0.0 from 10.10.30.73, dev eth1.30"

2. test with DNS 8.8.8.8/1.1.1.1 allowed, but same result:

 "2024-04-29 11:06:46","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6320 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:06:43","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6319 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:06:40","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6316 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:06:37","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6311 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:06:34","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6310 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:06:31","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6307 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:06:29","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=47.91.91.232 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6306 PROTO=TCP SPT=62204 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:05:01","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:86 SRC=10.10.30.73 DST=8.211.32.133 LEN=134 TOS=0x06 PREC=0x00 TTL=254 ID=6284 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH FIN URGP=0 "
"2024-04-29 11:05:01","warning","kern","kernel","ubnt"," [WAN_OUT-2004-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6283 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:46","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6260 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:43","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6257 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:41","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6255 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:37","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6243 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:34","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6241 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:31","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6230 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:28","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:30 SRC=10.10.30.73 DST=8.211.32.133 LEN=48 TOS=0x06 PREC=0x00 TTL=254 ID=6222 PROTO=TCP SPT=62203 DPT=10081 WINDOW=8760 RES=0x00 SYN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:25","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:86 SRC=10.10.30.73 DST=8.211.32.133 LEN=134 TOS=0x06 PREC=0x00 TTL=254 ID=6217 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH FIN URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:25","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6216 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:03:22","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:86 SRC=10.10.30.73 DST=8.211.32.133 LEN=134 TOS=0x06 PREC=0x00 TTL=254 ID=6209 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH FIN URGP=0 MARK=0x65000000 "
"2024-04-29 11:02:37","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6141 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:02:13","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6111 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:02:01","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6098 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:01:55","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6090 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:01:52","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6083 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
"2024-04-29 11:01:51","warning","kern","kernel","ubnt"," [WAN_OUT-2003-D]IN=eth1.30 OUT=eth2 MAC=xx:xx:xx:xx:57 SRC=10.10.30.73 DST=8.211.32.133 LEN=87 TOS=0x06 PREC=0x00 TTL=254 ID=6079 PROTO=TCP SPT=62202 DPT=10081 WINDOW=7880 RES=0x00 ACK PSH URGP=0 MARK=0x65000000 "
DirkBaumeister commented 5 months ago

Interesting! Thank you so far for testing, this looks really good for debugging (Even when I don't own one yet :D) Any chance to log which dns requests are done here? Are you blocking or rejecting the request? I could imagine that a REJECT would work better here because it tells the device "Don't wait for a response, nothing to fetch here, abort". (If you don't already do so :) )

N3rdix commented 5 months ago

It indeed was a DROP, I can log another test with REJECT and see if it makes a difference.

As it bypasses my own DNS (Adguard) I can't directly see which requests the device initiates. If I see something related in my gateway's logs I'll attach them as well.

DirkBaumeister commented 5 months ago

Thank you! I don't know what firewall you are using but it is possible to "bend" requests to DNS via iptables for example to route them to your own dns server. An example can be found here: https://askubuntu.com/a/592398

EDIT: I don't think you need to modify your firewall. The requested DNS record is dataeu.hoymiles.com: Screenshot

N3rdix commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

IIChrisII commented 5 months ago

I think we have to differenciate between human observation by watching to sensors in Home Assistant and concret data provided by the device. If you look to the log of the device in Home Assistant all sensors of the HMS gets unavailable and then comes back. This produces the 0-Values for a short time. My conclusion (as described above) is, that the DTU (which is only the communication interface) tries to send Data to the hoymiles cloud service (the fixed dns ip is already known), this blocks due to the firewall rule but HMS is waiting for connection timout and this makes the complete interface unavailable -> Home Assistant Sensors are going unavailable -> logger is writing 0 Values. Then Connection Timout occurs and DTU responds to requests from HA again -> correkt values are shown again.

While DTU ist waiting for connection to be established, home assistant sensors get unavailable/0. The Inverter (not DTU) is still working and producing korrekt values but these can not be requested because of the DTU. (This is just a logical assumption which is supported by the Log of the DTU in Home assistant. At the time, the readings get 0 you will see, the dtu is disconnected, then comes back).

IIChrisII commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

suaveolent commented 5 months ago

I think we have to differenciate between human observation by watching to sensors in Home Assistant and concret data provided by the device. If you look to the log of the device in Home Assistant all sensors of the HMS gets unavailable and then comes back. This produces the 0-Values for a short time. My conclusion (as described above) is, that the DTU (which is only the communication interface) tries to send Data to the hoymiles cloud service (the fixed dns ip is already known), this blocks due to the firewall rule but HMS is waiting for connection timout and this makes the complete interface unavailable -> Home Assistant Sensors are going unavailable -> logger is writing 0 Values. Then Connection Timout occurs and DTU responds to requests from HA again -> correkt values are shown again.

While DTU ist waiting for connection to be established, home assistant sensors get unavailable/0. The Inverter (not DTU) is still working and producing korrekt values but these can not be requested because of the DTU. (This is just a logical assumption which is supported by the Log of the DTU in Home assistant. At the time, the readings get 0 you will see, the dtu is disconnected, then comes back).

This perfectly summarises the problem. As I said, I could implement a workaround to "fix" the graph, but if you are using an automation based on the value reported, it will only report an outdated value and not the actual value.

DirkBaumeister commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

Would be cool to see what the inverter does if someone could response to dns request for dataeu.hoymiles.com with the ip address 0.0.0.0, like pihole does. This should result in an abort of the request hopefully.

suaveolent commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

Would be cool to see what the inverter does if someone could response to dns request for dataeu.hoymiles.com with the ip address 0.0.0.0, like pihole does. This should result in an abort of the request hopefully.

This was also an idea I had for the firmware update. Another thing we could try: The get-config request reports back:

server_domain_name: "dataeu.hoymiles.com"

We could even try setting that value to our Home Assistant instance and then provide a response. However, this still means bending the DNS responder.

DirkBaumeister commented 5 months ago

Yes, absolutely. Is the inverter somewhat operational even without panels attached? So maybe someone (Or me, if I get my hands on one) could debug this even without going "full solar installation" already.

Edit: Don't think so because it only is accessible when there is sun (= panels attached) right?

suaveolent commented 5 months ago

Yes, absolutely. Is the inverter somewhat operational even without panels attached? So maybe someone (Or me, if I get my hands on one) could debug this even without going "full solar installation" already.

Edit: Don't think so because it only is accessible when there is sun (= panels attached) right?

Yes, you need panels and/or battery.

DirkBaumeister commented 5 months ago

Too bad :D Hmm. Maybe I can provide an internet "fake endpoint" somehow later. @suaveolent Any chance to test your idea of changing the api endpoint on your test unit? This wouldn't require dns rewrite at all because it would be a public server with valid ssl certificates. If so, please let me know, than I can send you an endpoint (personally somehow) :)

IIChrisII commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

Would be cool to see what the inverter does if someone could response to dns request for dataeu.hoymiles.com with the ip address 0.0.0.0, like pihole does. This should result in an abort of the request hopefully.

i would try this. But actually im struggeling because i block the communication simply by fritzbox. How do I ensure that DNS requests for 8.8.8.8 are redirected to my DNS server but the rest of the communication from the device is blocked?

DirkBaumeister commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

Would be cool to see what the inverter does if someone could response to dns request for dataeu.hoymiles.com with the ip address 0.0.0.0, like pihole does. This should result in an abort of the request hopefully.

i would try this. But actually im struggeling because i block the communication simply by fritzbox. How do I ensure that DNS requests for 8.8.8.8 are redirected to my DNS server but the rest of the communication from the device is blocked?

That's not so trivial if you haven't worked with iptables or similar before

DirkBaumeister commented 5 months ago

@suaveolent You can actually use something like https://webhook.cool This would provide you an endpoint, for example: https://abc.webhook.cool which you can (hopefully) set as api endpoint

IIChrisII commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

Would be cool to see what the inverter does if someone could response to dns request for dataeu.hoymiles.com with the ip address 0.0.0.0, like pihole does. This should result in an abort of the request hopefully.

i would try this. But actually im struggeling because i block the communication simply by fritzbox. How do I ensure that DNS requests for 8.8.8.8 are redirected to my DNS server but the rest of the communication from the device is blocked?

That's not so trivial if you haven't worked with iptables or similar before

have done this before but it's definetly not my daily business and im not an (network)administrator. Maybe you can help. DNS Server is set up to answer with 0.0.0.0 for "dataeu.hoymiles.com". Now i woul set up a static route for 8.8.8.8 to my dns server, right? What is the network/subnetmask of 8.8.8.8?

DirkBaumeister commented 5 months ago

same result with REJECT as for DROP. My old Unifi gateway (USG) does not support DNS "masquerading", so no more logs from my side

Do i correctly understand that you rejected (not blocked) somehow the ip connections to 8.8.8.8?

Would be cool to see what the inverter does if someone could response to dns request for dataeu.hoymiles.com with the ip address 0.0.0.0, like pihole does. This should result in an abort of the request hopefully.

i would try this. But actually im struggeling because i block the communication simply by fritzbox. How do I ensure that DNS requests for 8.8.8.8 are redirected to my DNS server but the rest of the communication from the device is blocked?

That's not so trivial if you haven't worked with iptables or similar before

have done this before but it's definetly not my daily business and im not an (network)administrator. Maybe you can help. DNS Server is set up to answer with 0.0.0.0 for "dataeu.hoymiles.com". Now i woul set up a static route for 8.8.8.8 to my dns server, right? What is the network/subnetmask of 8.8.8.8?

Hmmm, I don't think a static route will help you here. You have to modify the request via iptables and DNAT in the PREROUTING process that it "bends" the request to 8.8.8.8 to your own ip address (Like mentioned here: https://askubuntu.com/a/592398 )

EDIT: A basic rule would look like this:

iptables -A PREROUTING ! -s <DNS SERVER IP>/32 ! -d <DNS SERVER IP>/32 -p tcp -m tcp -m iprange --src-range <IP RANGE START>-<IP RANGE END> --dport 53 -j DNAT --to-destination <DNS SERVER IP>:53
iptables -A PREROUTING ! -s <DNS SERVER IP>/32 ! -d <DNS SERVER IP>/32 -p udp -m udp -m iprange --src-range <IP RANGE START>-<IP RANGE END> --dport 53 -j DNAT --to-destination <DNS SERVER IP>:53

DNS SERVER IP = The ip address of your dns server IP RANGE START = The first ip of client that should be modified IP RANGE END = The last ip of client that should be modified

IIChrisII commented 5 months ago

Pity. I don't think I can do that with the FritzBox ;).