Add offset criteria for self-regulation when in over_climate which underlying that have a temperature sensor

[ikoz]

Version of the custom_component

5.2.2

Configuration

hvac_modes:
  - auto
  - heat
  - "off"
min_temp: 7
max_temp: 35
target_temp_step: 0.5
preset_modes:
  - none
  - frost
current_temperature: 17.5
temperature: 19.2
hvac_action: idle
preset_mode: none
is_on: true
hvac_mode: heat
type: null
is_controlled_by_central_mode: false
last_central_mode: null
frost_temp: 5
eco_temp: 0
boost_temp: 0
comfort_temp: 0
frost_away_temp: 7
eco_away_temp: 7
boost_away_temp: 7
comfort_away_temp: 7
power_temp: null
target_temperature_step: 0.5
ext_current_temperature: -3.31
ac_mode: false
current_power: null
current_power_max: null
saved_preset_mode: none
saved_target_temp: 19.2
saved_hvac_mode: null
window_state: "off"
motion_state: null
overpowering_state: null
presence_state: null
window_auto_state: "off"
window_bypass_state: false
security_delay_min: 60
security_min_on_percent: 0.5
security_default_on_percent: 0.1
last_temperature_datetime: "2024-01-18T09:23:38.671679+00:00"
last_ext_temperature_datetime: "2024-01-18T09:15:23.759238+00:00"
security_state: false
minimal_activation_delay_sec: 10
device_power: 1
mean_cycle_power: null
total_energy: 0.13
last_update_datetime: "2024-01-18T09:23:38.676592+00:00"
timezone: Europe/London
window_sensor_entity_id: null
window_delay_sec: null
window_auto_enabled: false
window_auto_open_threshold: null
window_auto_close_threshold: null
window_auto_max_duration: null
motion_sensor_entity_id: null
presence_sensor_entity_id: null
power_sensor_entity_id: null
max_power_sensor_entity_id: null
temperature_unit: °C
is_device_active: false
ema_temp: 17.45
is_over_climate: true
start_hvac_action_date: null
underlying_climate_0: climate.wiser_dinning_room
underlying_climate_1: null
underlying_climate_2: null
underlying_climate_3: null
is_regulated: true
regulated_target_temperature: 22.1
auto_regulation_mode: auto_regulation_strong
regulation_accumulated_error: 28.109999999999992
auto_fan_mode: auto_fan_none
current_auto_fan_mode: auto_fan_none
auto_activated_fan_mode: null
auto_deactivated_fan_mode: null
friendly_name: Dinning Room Thermostat
supported_features: 17

If it is releveant to regulation performance or optimisation some curves are needed

I'm attaching the following picture to show the problem. In this plot:

• PURPLE line: My desired target temperature (this has been set as target temperature in VTech climate entity)
• DARK BLUE line: The accurate temperature of the room (external zigbee sensor). This is the sensor set in VTech as external sensor.
• LIGHT BLUE line : The internal TRV temperature sensor - takeng from the TRV climate entity.
• ORANGE line: The TRV setpoint, configured by VTech (regulated temperature).
• DARK ORANGE vertical bars: the duration where the TRV was open and heating

Describe the bug

I'm was trying to heat the room to 17.2C and then 19.2C (from 6:30AM onwards) (purple line - vtech entity target temperature) The real temperature of the room dropped below 17.2C around 3:00AM. (dark blue line) You can see in the graph that around 3:00AM, Vtech started raising the TRV setpoint (regulated temperature - orange line).

However, the TRV did not actually open to let heat into the room, because the internal TRV temperature (light blue line) was higher than the regulated TRV setpoint (orange line) - or it jumped to immediately be higher after heating for a few minutes.

Then I raised the target temperature to 19.2C around 6.30AM. The same thing happened.

So Vtech has been trying to heat the room since 3.00AM and it is now 10AM and the room has not been heated, because the TRV has not actually opened for more than a few short bursts that each lasts a few minutes (vertical bars). These short bursts are also happening at "low demand %", since the demand % is the difference between regulated temperature setpoint and internal trv temperature. This is important, because I am using opentherm: Low demand % means low flow temperature, meaning very slow heating response.

I believe the Vtech self-regulation algorthm should take into account the internal TRV temperature of the climate entity, and calculate an "offset" based on the current value of the internal TRV temperature, so that the regulated temperature is above the internal TRV temperature, and the TRV lets water to flow.

For example I'd expect that if real room temperature is 16C, target temperature is 19C and internal trv temperature is 22C, then Vtech should set the regulated temperature to 25C (a lot higher than 22C, since 19C target is much higher than the 16C real temperature)

Let me know if you need a debug log, but it is fairly obvious what is happening here. VTech calculates the regulated temperature based on external room sensor, completely ignoring the internal TRV sensor. Even in 'hard' self-regulation mode, it cannot work. I am not sure it would work in expert mode either.

[ikoz]

@maia when self regulation is used with TRVs that have internal temperature sensors, sometimes the system does not work as expected.

The new feature is aimed to make the self regulation system to effectively ignore any bad effects caused by the internal TRV sensor.

[jmcollin78]

@ikoz ,

The new feature is aimed to make the self regulation system to effectively ignore any bad effects caused by the internal TRV sensor.

Yes. Exactly. Your explanation is better than mine.

If I can resume the algorithm:

1. if self regulation is on, a. calculate the regulated temp (one per VTherm),
2. else take target_temp as regulated_temp
3. for each TRV of the VTherm, a. if "use internal temp" is on, i. calculate the offset (trv internal temp - room temp), b. else offset=0 c. add the offset to the regulated temp, d. send the regulated temp (+offset then) to the TRV.

Yes. When room temp is above target, there is no need of regulation nor offset calculation.

Why? If it's too warm in a room, I want the TRV to have a lower target temperature set. I want to minimise my heating gas usage. So if a room is warmed by the sun or the oven, making the room temperature be higher than the TRV temperature, I want the climate entity to send a lower temperature to the TRV. I thought that's what this feature is about, no?

That is the case. My explanation was confusing. Look at this graph:

The regulated temp is 2.5° under the target because the room temp is > target for a long time. when target goes to 17°, the regulated temp decrease to 5° under the target (due to Strong regulation).

I hope it is more clear now.

[jmcollin78]

Merged: https://github.com/jmcollin78/versatile_thermostat/pull/373

[jmcollin78]

https://github.com/jmcollin78/versatile_thermostat/releases/tag/5.4.1

[ikoz]

Quite interesting behavior this morning. The positive is that the system behaved correctly and heating came on fast. Not sure if the trv setpoint should be so much higher compared to regulated temperature or the trv temp. I think I'd expect the offset to be smaller

[jmcollin78]

@ikoz ,

TRV internal temp is +4° of room temp, so the set-point is regulated + 4°. I agree that +2° should be sufficient to force heating. Because this value is limited by the max_temp configuration value, I don't think it is an issue.

[ikoz]

@ikoz ,

TRV internal temp is +4° of room temp, so the set-point is regulated + 4°. I agree that +2° should be sufficient to force heating. Because this value is limited by the max_temp configuration value, I don't think it is an issue.

One aspect of this that I have not mentioned in this thread before is that I have an opentherm system. This means that the system operates proportionally to demand. If demand is 100% then the valve is fully open, if demand is 50% then valve is half open.

Demand is calculated based on the difference between trv setpoint and trv internal temperature.

If trv setpoint is always much higher than trv internal temperature, then demand is always 100%. This makes the system inefficient. The expected behavior is at the start of heating to be high demand and the lower. The boiler flow temperature is also controlled based on demand.

Basically now the next stage is to figure out how to make the system behave a bit better by keeping the (trv setpoint - trv temp) proportional to (vtherm setpoint - real room temp). I'll try to plot these two differences later today and monitor them.

[jmcollin78]

This should not be an issue. As soon as the room temp will approach the target, the regulation will decrease. Then the target send to TRV will decrease also. One the target is reach, there is no more internal temp offset calculated. So it should work out of the box.

I have a pac with the same kind of feature. And it works well. The only drawback I see until now, is that target is send mode frequently to the pac and it is bipping each time, which is not very comfortable when the pac is in the bedroom.

[maia]

Thanks everyone. I think this feature does not solve my issue of overheating. Some TRVs have an internal sensor that measures always too high. I was hoping that the feature that compares the room temperature with the TRV temperature will notice this difference and adapt the (regulated) target temperature. But that's still not the case. Most of the case all three lines are the same (TRV target, VTherm target, VTherm regulated):

That's unfortunate as I really want to consume less heating gas and I hate that some rooms are always too warm. I know, I can turn down the temperature manually, but isn't the concept about all this self regulating that I don't need to keep in mind things like "this room is always 1°C too warm, so I will set the target lower. And this other room is always too cold, so I will increase the target".

[jmcollin78]

@maia this should really do what you expect. Can you please post the attributes of the related VTherm ?

Did you activate self-regulation for this VTherm ? I guess not.

[maia]

@maia this should really do what you expect. Can you please post the attributes of the related VTherm ?

Did you activate self-regulation for this VTherm ? I guess not.

Here are the attributes for this VTherm:

hvac_modes:
  - "off"
  - heat
min_temp: 14
max_temp: 26
target_temp_step: 0.1
preset_modes:
  - none
  - frost
  - eco
  - comfort
  - boost
current_temperature: 20.9
temperature: 20
hvac_action: heating
preset_mode: comfort
is_on: true
hvac_mode: heat
type: null
is_controlled_by_central_mode: true
last_central_mode: Auto
frost_temp: 12
eco_temp: 19
boost_temp: 21
comfort_temp: 20
frost_away_temp: 12
eco_away_temp: 18
boost_away_temp: 20
comfort_away_temp: 19
power_temp: 13
target_temperature_step: 0.1
ext_current_temperature: 10.3
ac_mode: false
current_power: null
current_power_max: null
saved_preset_mode: comfort
saved_target_temp: 20
saved_hvac_mode: null
motion_sensor_entity_id: null
motion_state: null
power_sensor_entity_id: null
max_power_sensor_entity_id: null
overpowering_state: null
presence_sensor_entity_id: input_boolean.presence_family
presence_state: "on"
window_state: "off"
window_auto_state: "off"
window_bypass_state: false
window_sensor_entity_id: null
window_delay_sec: 120
window_auto_enabled: true
window_auto_open_threshold: 6
window_auto_close_threshold: 0
window_auto_max_duration: 60
window_action: window_turn_off
security_delay_min: 60
security_min_on_percent: 0.5
security_default_on_percent: 0.1
last_temperature_datetime: "2024-02-04T13:37:31.628972+01:00"
last_ext_temperature_datetime: "2024-02-04T13:38:02.067762+01:00"
security_state: false
minimal_activation_delay_sec: 10
device_power: 1
mean_cycle_power: null
total_energy: 1478.3
last_update_datetime: "2024-02-04T13:38:02.090267+01:00"
timezone: Europe/XXX
temperature_unit: °C
is_device_active: true
ema_temp: 20.89
is_used_by_central_boiler: false
is_over_climate: true
start_hvac_action_date: "2024-02-04T11:13:26.388844+01:00"
underlying_climate_0: climate.arbeitszimmer_eve_trv
underlying_climate_1: null
underlying_climate_2: null
underlying_climate_3: null
is_regulated: true
regulated_target_temperature: 20
auto_regulation_mode: auto_regulation_light
regulation_accumulated_error: -10
auto_fan_mode: auto_fan_none
current_auto_fan_mode: auto_fan_none
auto_activated_fan_mode: null
auto_deactivated_fan_mode: null
auto_regulation_use_device_temp: true
friendly_name: Arbeitszimmer Heizung
supported_features: 17

and here is a current screenshot:

(and thanks for looking into it!)

[jmcollin78]

@maia ,

You have light_regulation. So you have this configuration now:

class RegulationParamLight:
    """Light parameters for regulation"""

    kp: float = 0.2
    ki: float = 0.05
    k_ext: float = 0.05
    offset_max: float = 1.5
    stabilization_threshold: float = 0.1
    accumulated_error_threshold: float = 10

The regulation_accumulated_error is at top (-10). The offset is -10 x 0.05 -> 0.5°.

I don't see the auto_regulation_threshold (maybe it is missing in the attributes), but I guess it is 0.5 or less so the regulation is never applied with this configuration.

2 options: change to a higher regulation level (minimum Medium because you need at least 1°) or change the regulation threshold. I suggest you to set the High self-regulation level and check the value of threshold.

[maia]

The regulation_accumulated_error is at top (-10). The offset is -10 x 0.05 -> 0.5°.

Thanks, I didn't know that the "light" setting doesn't do anything for TRVs, which mostly have a 0.5°C step regulation as far as I know. Yesterday I then switched to "strong" by your suggestion, but that's a terrible over-regulation that's not usable for TRVs. Ideally a TRV holds the equilibrium of valve position depending on the outdoor temperature, instead of opening to 100% until the room starts to overheat and then down to 0% until it's too cool. Which is why I still think a proper long-term offset definition would help.

I've now switched to "slow" and if that does not work, I will switch to my super slow settings in expert mode. But judging by the graphs of the past days, the new feature doesn't do much in my case. I had the wrong assumption that you were implementing a long term offset (long term difference room temp and trv temp).

Please consider implementing my long suggested feature of long term offset calculation by taking the (at least) 1-day average difference of room temp and TRV temp.

[jmcollin78]

@maia,

Isn't it good now ? I see blue line and red line are following each other.

Ideally a TRV holds the equilibrium of valve position depending on the outdoor temperature, instead of opening to 100% until the room starts to overheat and then down to 0% until it's too cool.

Can you display the valve open percent ? I don't know what makes you think the valve are only open to 0% and 100%. In stable situation (Arbeitzimmer from 0:00 -> 6:00), I think you have small heating time when room temp falls under the target. What is wrong in your opinion ?

I talk about this time frame:

I consider calculating a long term offset is useless for most of the cases. See my example:

Can you tell me what is the long term offset and what can I do with the value I will find ?

If I look at yours, can you tell me the long term offset on this curves ?

It is certainly near 0. On the first part it is >> 0 and on the second part it is < 0.

I know you are claiming this long term offset for a long time, but in my opinion:

1. this is useless as we don't need that value,
2. this is wrong. The values will be always wrong and not usable for regulation nor forcing heating. Or please explain me how I can use this value.

What I see here is a good regulation (the blue follow the red) considering the latency of your heating system:

[maia]

@jmcollin78 Thanks, much appreciated. As for the long term offset, I will start calculating the 24 hours average room temp and trv temp and will provide a chart in a few days. I'm curious myself.

Regarding the overregulation causing the valves to fully close or open: The temperature humans feel consists of (at least) two aspects: 1) the temperature of the air and 2) the temperature of the surfaces around you. The latter is why people like sitting around a campfire at night. The air temperature is low, but the fire has a high amount of radiation, causing it to feel warm.

So if a valve closes completely, the radiator cools down from e.g. 30-40°C to room temperature. This reduces the radiation and also causes the cold air at the windows to drop down to the floor, causing a cycle of cold air flowing into the room. If a radiator keeps a minimum temperature of e.g. 30°C, this won't happen and it will feel much more comfortable.

While the "strong" regulation might minimise the difference between room temperature and target temperature, it does not feel warm. If feels hot when the radiator heats up and the feels cold an hour later, when it has been at 0% for a while.

This is why I believe that ideally a valve position will only change minimally over the day and it should stay within a range of 10% to 80%. Smart TRVs handle accordingly. But the VTherm over-regulation of the "strong" setting works against that.

I've switched to "slow" regulation and am watching it now for a few days. But I might go back to my super-slow setting in expert mode.

Here's the valve opening of the time frame requested:

[jmcollin78]

Hello @maia ,

So in stable situation, the oscillation you observe is between 0%-30%. I agree the best thing to achieve is to have something stable, let's say at 15% in the Arbeitszimmer room. So I'm curious if you find some tuning / method to achieve this.

In the TPI algorithm, this is typically a coef_int which is too high. But in your case, the real regulation is done by the TRV itself and VTherm "only" send a adapted target temp. So if the TRV itself have a bad regulation algorithm or a bad temp sensor (and it seems it is the case), i'm not sure we will have the possibility to fix those oscillations.

If this is really a problem for you, the best think you can do is to change your TRV, and buy some TRV in which VTherm will be able to control directly the open percent of the valve. So you will use the TPI algorithm which is simple to tune and very efficient (from what I can see from other uses report - I don't have this myself).

Shell TRV does this for example (but the price is something, I agree).

[maia]

Hi @jmcollin78, as a quick update: Three days ago I have created 24 hours average sensors for each the TRV temperature and the room temperature for two rooms:

sensor:
  - platform: average
    name: "Schlafzimmer 24hrs average room temperature"
    unique_id: "schlafzimmer_24hrs_average_room_temperature"
    entities:
      - sensor.schlafzimmer_govee_temperatursensor_temperature
    duration:
      hours: 24
    precision: 2

So now, after two full days: As a big surprise, even with a 24 hours average there is no constant offset between TRV internal temperature and room temperature:

The only two explanations I can come up with is that this offset depends on the outdoor temperature and also if I'm at home during the day or away at the office. I'm going to continue to watch this difference.

So in stable situation, the oscillation you observe is between 0%-30%. I agree the best thing to achieve is to have something stable, let's say at 15% in the Arbeitszimmer room. So I'm curious if you find some tuning / method to achieve this.

I am currently using the "slow" self-regulation but am considering using a slower setting:

versatile_thermostat:
  auto_regulation_expert:
    kp: 0.1 # 10% of the current internal regulation offset are caused by the current difference of target temperature and room temperature
    ki: 0.003125 # 0.9 / 288.0 = 90% of the current internal regulation offset are caused by the average offset of the past 24 hours
    k_ext: 0 # disable for now
    offset_max: 4.0 # limit to a final offset of -4°C to +4°C
    stabilization_threshold: 0.0 # this needs to be disabled as otherwise the long term accumulated error will always be reset when the temp briefly crosses from/to below/above the target
    accumulated_error_threshold: 1152 # 3.0 * 288 = this allows up to 4°C long term offset in both directions

You recently mentioned that stabilization_threshold has been removed, did you re-introduce it with the latest version to fix the issues with changing prefix (moving from too cold to too warm)? Do I need to keep it or do I need different configuration settings in the yaml file?

If this is really a problem for you, the best think you can do is to change your TRV, and buy some TRV in which VTherm will be able to control directly the open percent of the valve. So you will use the TPI algorithm which is simple to tune and very efficient (from what I can see from other uses report - I don't have this myself).

I'm looking into replacing all my 12 TRVs this autumn with devices that support zigbee2mqtt as I'm moving my entire smart home to this standard. Unsure which ones to buy – I want them silent, compact (otherwise they won't fit), with a display on the side and not the top (as otherwise the display points towards the wall and cannot be seen) and with a good temperature regulation, so there aren't any issues when HA is down for some reason.

So I'm slowly saying goodbye to Thread and HomeKit, at least for the next 24 months.