Closed beniwohli closed 2 years ago
I also noticed this behavior. It started after upgrading the firmware in the bridge to version U1.2.6. Additionally, the new firmware version allows up to 5 simultaneous connections. According to the release notes, the stability has also been improved.
I also notice this since I updated the app (and that updated the firmware). I'll probably need to filter out those "bad" values somehow.
Same issue here. Yesterday I upgraded my ComfoConnect. Since then I see the same behaviour as described.
It would be great to have this fixed!
any plans for a new release with this fix?
Hi, this is related to the DHCP lease time. The new firmware seems to be closing the connections whenever it receives a new lease from the DHCP. So for example: If you've set the DHCP lease time to 60 minutes, the device will request a new IP every 30 minutes and thus you will see a zero-value in your graph every 30 minutes. Try to increase the lease time to something bigger such as a day. You should be seeing less of the zero values. I'm in contact with the vendor about this already, from my point of view this should be fixed in the firmware.
I have manually assigned IP so is not changing
It is not about the IP changing, it is about the lease time. Even in the case of a static assignment the lease has a validity which is set by the DHCP server. Once the lease expires, the client needs to request a new lease and that causes the trouble. Run a tcpdump on your DHCP server or switch and try to correlate the time of the zero values with the DHCP Request / DHCP ACK messages you see for the gateway. Then you will see what I mean. You can also try to capture port 56747 on the gateway. You will see that the connection is closed by the gateway at the same time the DHCP Request is happening.
Hi, this is related to the DHCP lease time. The new firmware seems to be closing the connections whenever it receives a new lease from the DHCP. So for example: If you've set the DHCP lease time to 60 minutes, the device will request a new IP every 30 minutes and thus you will see a zero-value in your graph every 30 minutes. Try to increase the lease time to something bigger such as a day. You should be seeing less of the zero values. I'm in contact with the vendor about this already, from my point of view this should be fixed in the firmware.
I have also opened a ticket for this, but I'm not sure if this has any priority at Zehnder. It would be nice if you could keep us up to date if you have more info in your ticket.
I agree that this issue should be fixed in the firmware itself. It's an issue that wasn't present in the original firmware.
Will do, I have a call with them on Friday.
I'm somewhat optimistic that they will fix this issue; I figured out that the official App is also affected by the issue. When you are using it (for example viewing the numbers on the status page) and the lease is renewed the connection of the App is lost and an error message is displayed. A reconnect is possible, but only manually. So it should be in their interest as well to fix this. Let's see.
So, I just had the call with Ercan (you might know him, Michaël, he mentioned you talked as well). He agreed that this is an error within the firmware and it will be corrected by the developers. However, he also mentioned that this might take quite some time, since the quality assurance process on any new firmware release for the gateways is lengthy. He mentioned one interesting fact: It seems to be normal that the first value send by the gateway on any new connection is always zero. The zero values will continue to be send on any new connection also in the future. What needs to be fixed in the firmware is the termination of open connections during DHCP lease renewal.
I will not apply the workaround suggested in the pull request to my setup, instead I just set the lease time to a really large value for the time being. Once I receive the new firmware I will test this again and let you know my findings.
He mentioned one interesting fact: It seems to be normal that the first value send by the gateway on any new connection is always zero. The zero values will continue to be send on any new connection also in the future.
That's interesting, I can't remember seeing these zero values in earlier firmware versions. But if their plan is to continue sending these values, I think for at least power_total
, a filter is necessary to use the data for the energy dashboard. Otherwise, HA will think that the meter has been reset when a 0 is recorded, and then assume that the appliance used X kwh in an instant (at least that's how I understand how the energy dashboard and the statistics calculation works).
I did speak with him indeed. I also don't remember seeing these 0 values, and I still have a few old traffic dumps. I'm quite sure that this behaviour has been introduced in the new firmware version.
About the power consumption. I actually use the "integration" feature of home assistant to calculate a total based on the actual value since comfoconnect returns the totals in KWh, and it can take a few days to use a KWh when on a low fan speed.
@hanjo @beniwohli
I'm really struggling to find a good solution here. We can't just ignore the first value if it's zero, since some sensors like days to replace sensor
can report a valid 0 value, and when we ignore the first value, we won't get a second update, so the value will stay undefined forever.
I'm trying to find the actual cause of the bug, maybe waiting a bit before we register the sensors can help, but I can't reproduce the issue besides waiting a long time until it happens.
I rely on the ComfoConnect values to control my ventilation. Since the firmware update I experienced problems. I finally ended using the commited code, combined with these "filtered" sensors. In the recorder, I excluded all ComfoAirQ sensors. It may be temporary, but at least, this works for me. I hope this code helps some of you...
recorder:
exclude:
entities:
- sensor.comfoairq_exhaust_humidity
- sensor.comfoairq_inside_humidity
- sensor.comfoairq_outside_humidity
- sensor.comfoairq_supply_humidity
- sensor.comfoairq_power_usage
- sensor.comfoairq_supply_temperature
- sensor.comfoairq_exhaust_temperature
- sensor.comfoairq_supply_airflow
- sensor.comfoairq_exhaust_airflow
- sensor.comfoairq_outside_temperature
- sensor.comfoairq_inside_temperature
- sensor.comfoairq_exhaust_fan_speed
- sensor.comfoairq_exhaust_fan_duty
- sensor.comfoairq_supply_fan_speed
- sensor.comfoairq_supply_fan_duty
- sensor.comfoairq_days_to_replace_filter
- sensor.comfoairq_bypass_state
sensor:
- platform: filter
name: "ComfoAirQ Supply temperature (filter)"
entity_id: sensor.comfoairq_supply_temperature
filters:
- filter: outlier
window_size: 4
radius: 2.0
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:05"
precision: 1
- platform: filter
name: "ComfoAirQ Exhaust temperature (filter)"
entity_id: sensor.comfoairq_exhaust_temperature
filters:
- filter: outlier
window_size: 4
radius: 2.0
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:05"
precision: 1
- platform: filter
entity_id: sensor.comfoairq_outside_temperature
name: "ComfoAirQ Outside temperature (filter)"
filters:
- filter: outlier
window_size: 4
radius: 2.0
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:05"
precision: 1
- platform: filter
name: "ComfoAirQ Inside temperature (filter)"
entity_id: sensor.comfoairq_inside_temperature
filters:
- filter: outlier
window_size: 4
radius: 2.0
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:05"
precision: 1
- platform: filter
name: "ComfoAirQ Exhaust humidity (filter)"
entity_id: sensor.comfoairq_exhaust_humidity
filters:
- filter: outlier
window_size: 4
radius: 4
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:05"
precision: 0
- platform: filter
name: "ComfoAirQ Inside humidity (filter)"
entity_id: sensor.comfoairq_inside_humidity
filters:
- filter: outlier
window_size: 4
radius: 4
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:01"
precision: 0
- platform: filter
name: "ComfoAirQ Outside humidity (filter)"
entity_id: sensor.comfoairq_outside_humidity
filters:
- filter: outlier
window_size: 4
radius: 4
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:01"
precision: 0
- platform: filter
name: "ComfoAirQ Supply humidity (filter)"
entity_id: sensor.comfoairq_supply_humidity
filters:
- filter: outlier
window_size: 4
radius: 4
- filter: lowpass
time_constant: 10
- filter: time_simple_moving_average
window_size: "00:01"
precision: 0
- platform: filter
name: "ComfoAirQ Bypass state (filter)"
entity_id: sensor.comfoairq_bypass_state
filters:
- filter: outlier
window_size: 4
radius: 100
precision: 0
- platform: filter
name: "ComfoAirQ Days to replace filter (filter)"
entity_id: sensor.comfoairq_days_to_replace_filter
filters:
- filter: outlier
window_size: 4
radius: 2
precision: 0
I think filtering is a good approach. Can you please explain what exactly this "outlier" filter means.
How you came up with different filters outlier
, lowpass
, time_simple_moving_average
and their properties?
Maybe a simple median filter^1 would be sufficient to filter out the peaks in each signal?
I think filtering is a good approach. Can you please explain what exactly this "outlier" filter means.
How you came up with different filters
outlier
,lowpass
,time_simple_moving_average
and their properties?Maybe a simple median filter1 would be sufficient to filter out the peaks in each signal?
Footnotes
The problem is that a filter like this isn't foolproof. Some sensors can return values close to zero, where an actual zero won't be filtered out since it's not out of place.
I think filtering is a good approach. Can you please explain what exactly this "outlier" filter means.
How you came up with different filters
outlier
,lowpass
,time_simple_moving_average
and their properties?Maybe a simple median filter1 would be sufficient to filter out the peaks in each signal?
Footnotes
1. https://www.mathworks.com/help/signal/ug/remove-spikes-from-a-signal.html [↩](#user-content-fnref-1-804bd97613c515b98d82493e2fe9a11e)
I found the explanation of the filtering here: https://www.home-assistant.io/integrations/filter/ Its all a bit fuzzy to me. I've based my config on trial an error an examples I found on the internet.
The most useful explanation of the 'oulier' (interpret as: 'what lies out of a range') is: radius: defines the radius of the sphere that will be used for counting the neighbors (source: http://www.open3d.org/docs/latest/tutorial/Advanced/pointcloud_outlier_removal.html)
The only thing I know is that your changes, combined with these filtering, finally worked for me. I tried running without your commits, but this doesn't work out.
Filtering involves a few pitfalls, e.g. as the bypass opens (large fluctuation in bypass state, temperatures an humidity), but all is acceptable to me now.
I've abandoned the commits in the pull request, now using a 'stock' home assistant install (2021.12.9). I'm now successfully using the statistics platform, i.s.o. the filtering. Looks foolproof to me now... Fingers crossed.
e.g. sensor.comfoairq_outside_temperature:
e.g. sensor.comfoairq_days_to_replace_filter
Zehnder has finally pushed a new Comfoconnect app (for Android) that fixes a firmware update issue so it finally allows to upgrade the bridge to 1.5.1.
This firmware fixes the 0 values issue. I was running 1.3.0 before and had the 0 values issue.
How does one update the firmware? I seem to be running U1.2.6 on the ComfoConnect Gateway but I can't see any button in the app for firmware upgrades.
I I'm not mistaken, you need a Zehnder Cloud account. I created a support ticket at my local Zehnder Zendesk. For BE/NL: https://zehnderbe.zendesk.com/hc/nl-nl. Don't forget to mension your systems serial number. I once found this video: https://www.youtube.com/watch?v=GKLLCM5FFKg. The upgrade needs to be affected within your local network (same subnet).
How does one update the firmware? I seem to be running U1.2.6 on the ComfoConnect Gateway but I can't see any button in the app for firmware upgrades.
You'll need to install the latest beta app of Comfocontroll from the play store (released on 31th of august), when opening the app, the comfoconnect app displays a message to update the bridge. This indeed has to happen in the same subnet.
I was stuck on 1.3.0 for a long time, a bug in in the app prevented the upgrade.
Do you need you be part of the beta program? There is nog Zehnder Comfoconnect beta app in my Google Play store.
You need to be part of the beta indeed. But I thought only the firmware that gets pushed as part of the beta has this issue with the 0 values.
My ComfoAir Q runs R1.10.0, the ComfoConnect Gateway U1.2.6. This version suffers the 0-values bug. What can we expect from Zehnder? Will they release the new firmware (1.5.1) to non-beta testers?
They are indeed planning to also update non beta users, but I have no idea when this will happen.
I would suggest to try to join the beta. I think they have a page on their website about this somewhere.
So we have to wait for zehnder to release a new firmware. No idea when the updates will come. Hasn't anyone created a temporary solution (patching pycomfoconnect?) already, for those struggling with this for a long time already?
You could use a template sensor and filter out 0 values. I think I saw some examples passing by in the Comfoconnect topic on the Home Assistant forums.
I use the statistics sensor platform to filter out deviate values. Filtering out 0's will not work: e.g. outside temperature, bypass state, supply temperature, days to replace filter. These sensor will regularlytory become 0. I have tried using the filter platform (I posted example code above), but had better experience using the statistics sensor platform.
Do you have examples of the configuration using the statistics platform?
I use the below config for almost a year.
To filter the bypass state, I had to use two statistic filters: first value_max and afterwards median I use two automations (added below) to force a sensor update of the bypass state. This introduces a 50min delay. Better than nothing.
Don't forget to include "python_script:" in your configuration.yaml Create a "python_scripts" directory in your home assistant directory and create a file "force_update_state.py" with the below content.
force_update_state.py
entity_id = data.get('source')
current_state = hass.states.get(entity_id)
hass.states.set(entity_id, current_state.state, current_state.attributes, force_update=True)
configuration.yaml
- platform: statistics
name: "ComfoAirQ Supply temperature (filter)"
entity_id: sensor.comfoairq_supply_temperature
sampling_size: 5
state_characteristic: value_max
precision: 1
- platform: statistics
name: "ComfoAirQ Inside temperature (filter)"
entity_id: sensor.comfoairq_inside_temperature
sampling_size: 5
state_characteristic: value_max
precision: 1
- platform: statistics
name: "ComfoAirQ Exhaust temperature (filter)"
entity_id: sensor.comfoairq_exhaust_temperature
sampling_size: 7
state_characteristic: median
precision: 1
- platform: statistics
name: "ComfoAirQ Outside temperature (filter)"
entity_id: sensor.comfoairq_outside_temperature
sampling_size: 7
state_characteristic: median
precision: 1
- platform: statistics
name: "ComfoAirQ Exhaust humidity (filter)"
entity_id: sensor.comfoairq_exhaust_humidity
sampling_size: 5
state_characteristic: value_max
precision: 0
- platform: statistics
name: "ComfoAirQ Inside humidity (filter)"
entity_id: sensor.comfoairq_inside_humidity
sampling_size: 5
state_characteristic: value_max
precision: 0
- platform: statistics
name: "ComfoAirQ Outside humidity (filter)"
entity_id: sensor.comfoairq_outside_humidity
sampling_size: 5
state_characteristic: value_max
precision: 0
- platform: statistics
name: "ComfoAirQ Supply humidity (filter)"
entity_id: sensor.comfoairq_supply_humidity
sampling_size: 5
state_characteristic: value_max
precision: 0
- platform: statistics
name: "ComfoAirQ Bypass state (maxfilter)"
entity_id: sensor.comfoairq_bypass_state
sampling_size: 5
state_characteristic: value_max
precision: 0
- platform: statistics
name: "ComfoAirQ Bypass state (medianfilter)"
entity_id: sensor.comfoairq_bypass_state_maxfilter
sampling_size: 5
state_characteristic: median
precision: 0
- platform: statistics
name: "ComfoAirQ Days to replace filter (filter)"
entity_id: sensor.comfoairq_days_to_replace_filter
sampling_size: 5
state_characteristic: value_max
precision: 0
automations.yaml:
- alias: 'Force update sensor.comfoairq_bypass_state'
trigger:
platform: time_pattern
minutes: '/10'
action:
- service: python_script.force_update_state
data:
source: sensor.comfoairq_bypass_state
- alias: 'Force update sensor.comfoairq_bypass_state_maxfilter'
trigger:
platform: template
value_template: "{{ (now().minute - 1) % 10 == 0 }}"
action:
- service: python_script.force_update_state
data:
source: sensor.comfoairq_bypass_state_maxfilter```
Update: yesterday I had the chance to upgrade the ComfoConnect Gateway to 1.5.1 via the ComfoControl app. It seems the "drop-to-0" issue has been fixed... Fingers crossed!
Hi
I use comfoconnect via the home assistant integration. Looking at history graphs in home assistant, I realized that the values for all monitored sensors jumps to
0
regularly, about every 30 minutes:(the irregularities are from me trying to debug the issue).
After enabling DEBUG logging, this is what I see:
It looks like after a connection error, the first bunch of values received by the bridge are
0
, and the actual values are sent right after.This makes it difficult to implement e.g. automations on the
days_to_replace_filter
value. Also, the new energy dashboard in Home Assistant can't usepower_total
as the value isn't strictly monotonic (it would interpret every 0 value as a reset, then on the next update think that the device used X kWh instantly (X being the total number of kWhs used by the device).Does anybody else see this behavior?
Trying to think of work arounds, the pycomfoconnect could dismiss the first received value for each sensor after a reconnect (or even only dismiss it if it is
0
). If that is an acceptable solution, I can open a PR.