[schedy] Postprocessor for rounding values

[asjmcguire]

Schedy keeps saying that it is resending the value, due a missing confirmation and then gives up after 10 times. The problem is it DID get a confirmation, but obviously not the one that it is expecting - because of decimal places.

2020-07-10 15:30:03.200023 INFO schedy_heating: <-- [R:House] Value set to 19.299999999999997��.  [scheduled]
2020-07-10 15:30:03.240235 INFO schedy_heating: --> [R:House] [A:climate.house] Received value of 19.3��.
2020-07-10 15:30:33.009205 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:31:03.011664 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:31:33.017980 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:32:03.011147 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:32:33.010566 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:33:03.010735 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:33:33.010468 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:34:03.013450 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:34:33.009228 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:35:03.010387 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-10 15:35:33.010505 WARNING schedy_heating: !!! [R:House] [A:climate.house] Gave up sending value after 10 retries.

It behaves the same way with for example: 19.6000000007 (it will receive the confirmation of 19.6, but schedy must be expecting 19.6000000007 back). I did experiment with trying to add a postprocessor so that the final figure being sent to homeassistant was rounded to one decimal place using the round_to_step helper function, but I never managed to get it work, I kept getting an error compiling the expression messages.

For what it is worth, I am not doing any calculations that are complex enough to require multiple decimal places, the input temperature only has one decimal place, and all my calculations only add or subtract with one decimal place.

[asjmcguire]

Here is an example where it's not because the value has been rounded, it's purely because it's only receiving a temperature with 1 decimal place from the generic climate entity, but it sent a value with many more decimal places - and so it doesn't match.

2020-07-11 06:30:00.226393 INFO schedy_heating: --> Attribute 'state' of 'input_boolean.heating_night_mode' changed from 'on' to 'off', reevaluating <Room R:House>.
2020-07-11 06:30:01.265389 INFO schedy_heating: <-- [R:House] Value set to 19.400000000000002��.  [scheduled]
2020-07-11 06:30:01.382450 INFO schedy_heating: --> [R:House] [A:climate.house] Received value of 19.4��.
2020-07-11 06:30:01.414720 INFO schedy_heating: --> Attribute 'state' of 'sensor.cc_outside_temperature' changed from '8.6' to '8.3', reevaluating <Room R:House>.
2020-07-11 06:30:01.443420 INFO schedy_heating: --> Attribute 'state' of 'sensor.cc_30_min_gust' changed from '8.06' to '4.61', reevaluating <Room R:House>.
2020-07-11 06:30:31.010806 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:31:01.013356 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:31:31.022854 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:32:01.014103 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:32:31.011214 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:33:01.010965 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:33:31.012993 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:34:01.013292 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:34:31.009931 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:35:01.010698 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-11 06:35:31.009731 WARNING schedy_heating: !!! [R:House] [A:climate.house] Gave up sending value after 10 retries.

[asjmcguire]

And so you can see the actual calculations being performed - here is the relevant rules for the thermostat calculation:

    house:
      actors:
        climate.house:
      rescheduling_delay: 120
      watched_entities:
        - "input_boolean.heating_night_mode"
        - "input_boolean.home_state_home"
        - "sensor.cc_outside_temperature"
        - "sensor.cc_30_min_gust"
        - "sensor.cc_rain_rate"
      friendly_name: House
      schedule:
        - v: 18.5
          rules:
            - x: "Add(-0.2) if is_on('input_boolean.heating_night_mode') else Next()"
            - x: "Add(-0.3) if is_off('input_boolean.home_state_home') else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_30_min_gust')) > 35 else Next()"
            - x: "Add(+0.1) if (float(state('sensor.cc_outside_temperature')) < 15.1 and float(state('sensor.cc_rain_rate')) > 0) else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < -9.9 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < -4.9 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 0.1 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 5.1 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 10.1 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 14.1 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 30.1 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 24.9 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 20.9 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 16.5 else Next()"
            - { v: 19.2, start: "00:00", end: "02:00" }
            - { v: 19.2, start: "02:00", end: "04:00" }
            - { v: 19.2, start: "04:00", end: "06:00" }
            - { v: 19.2, start: "06:30", end: "08:00" }
            - { v: 19.4, start: "08:00", end: "10:00" }
            - { v: 19.4, start: "10:00", end: "12:00" }
            - { v: 19.4, start: "12:00", end: "14:00" }
            - { v: 19.4, start: "14:00", end: "16:00" }
            - { v: 19.4, start: "16:00", end: "18:00" }
            - { v: 19.4, start: "18:00", end: "20:00" }
            - { v: 19.4, start: "20:00", end: "22:00" }
            - { v: 19.4, start: "22:00", end: "00:00" }

[bob1de]

Hi,

These imprecise results basically occur due to the way floating point numbers are stored in computers.

A simple postprocessor should do the trick. Add it after all the Add() rules:

- x: "Postprocess(lambda result: round(result, 1) if isinstance(result, float) else result)"

The isinstance() check is necessary because you may want to return OFF from your schedule and OFF doesn't support rounding, so this postprocessor only rounds floats and just returns anything else as is.

Please tell me if it worked.

Best regards Robert

[bob1de]

BTW, the round_to_step() function is not appropriate in this case, actually it's an old artifact I've got to remove anytime soon. The built-in round() Python function does what you want.

[asjmcguire]

Hi,

These imprecise results basically occur due to the way floating point numbers are stored in computers.

A simple postprocessor should do the trick. Add it after all the Add() rules:

- x: "Postprocess(lambda result: round(result, 1) if isinstance(result, float) else result)"

The isinstance() check is necessary because you may want to return OFF from your schedule and OFF doesn't support rounding, so this postprocessor only rounds floats and just returns anything else as is.

Please tell me if it worked.

Best regards Robert

Fantastic, I don't know what I was doing wrong when it threw error messages, I was following the documentation. It isn't throwing any errors adding the postprocessor, so I will keep a watch on the logs and make sure it is only sending values with 1 decimal place. I can't be 100% certain, but it seemed like after it given up trying to send the value, if you manually changed the temperature (eg via Google), the 120 minute rescheduling delay was not respected and schedy changed the temperature again 15 minutes later (the next time a watched entity changed).

[asjmcguire]

BTW, the round_to_step() function is not appropriate in this case, actually it's an old artifact I've got to remove anytime soon. The built-in round() Python function does what you want.

One of these days, I will get round to starting one of the 10 Python courses I have in my Udemy account.....

[bob1de]

I can't be 100% certain, but it seemed like after it given up trying to send the value, if you manually changed the temperature (eg via Google), the 120 minute rescheduling delay was not respected and schedy changed the temperature again 15 minutes later (the next time a watched entity changed).

This is expected. Schedy can't know whether the manual temperature adjustment really was a manual adjustment or just another state update from the thermostat. I'll see if I can do something about it, but that's a different issue.

I'm closing this for now. Feel free to comment if the rounding didn't work as expected.

[asjmcguire]

Hi, sorry to bother you again. Sadly it doesn't appear to be working - I'm reposting just the rules - to check I have added the postprocessor to the correct place:

          rules:
            - x: "Add(-0.2) if is_on('input_boolean.heating_night_mode') else Next()"
            - x: "Add(-0.3) if is_off('input_boolean.home_state_home') else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_30_min_gust')) > 35 else Next()"
            - x: "Add(+0.1) if (float(state('sensor.cc_outside_temperature')) < 15.1 and float(state('sensor.cc_rain_rate')) > 0) else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < -9.9 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < -4.9 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 0.1 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 5.1 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 10.1 else Next()"
            - x: "Add(+0.1) if float(state('sensor.cc_outside_temperature')) < 14.1 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 30.1 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 24.9 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 20.9 else Next()"
            - x: "Add(-0.1) if float(state('sensor.cc_outside_temperature')) > 16.5 else Next()"
            - x: "Postprocess(lambda result: round(result, 1) if isinstance(result, float) else result)"

But as you can see the postprocessor doesn't appear to be rounding -

2020-07-12 10:45:03.139626 INFO schedy_heating: <-- [R:House] Value set to 19.299999999999997��.  [scheduled]
2020-07-12 10:45:03.173816 INFO schedy_heating: --> [R:House] [A:climate.house] Received value of 19.3��.
2020-07-12 10:45:33.012160 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-12 10:46:03.012756 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.
2020-07-12 10:46:33.012854 WARNING schedy_heating: !!! [R:House] [A:climate.house] Re-sending value due to missing confirmation.

[bob1de]

Could you please set debug: true and post the log of a schedule evaluation that leads to a not rounded value? It's these indented lines that show what rules are evaluated to what result I'm interested in, + the lines afterwards that tell the value is sent to the thermostat.

[bob1de]

My mistake, the postprocessor should look like this:

- x: "Postprocess(lambda result: round(result.value, 1) if isinstance(result.value, float) else result)"

By the time the postprocessor gets called, the result already is a Temp object, so it has to round the temperature's value instead of the Temp object itself.

I'll add rounding capabilities to Temp directly so that this will be more straightforward in the future.

[asjmcguire]

2020-07-12 11:27:42.051262 INFO schedy_heating: --- [R:House] Applying postprocessors.
2020-07-12 11:27:42.055720 INFO schedy_heating: --- [R:House] + Add(-0.1��)
2020-07-12 11:27:42.059786 INFO schedy_heating: --- [R:House] = 19.299999999999997��
2020-07-12 11:27:42.063833 INFO schedy_heating: --- [R:House] + <hass_apps.schedy.expression.types.Postprocess object at 0x7ff59c108040>
2020-07-12 11:27:42.067813 INFO schedy_heating: --- [R:House] = 19.3
2020-07-12 11:27:42.071936 INFO schedy_heating: --- [R:House] Final result: 19.3��, markers: set()
2020-07-12 11:27:42.075897 INFO schedy_heating: --- [R:House] Setting value to 19.3��.  [scheduled]
2020-07-12 11:27:42.080626 INFO schedy_heating: --- [R:House] [A:climate.house] Not sending value 19.3�� redundantly.

Thank you this appears to be working now.

[bob1de]

Great. Would you mind installing the latest development version from the master branch?

Just use this URL instead of "hass-apps" in your requirements.txt file or the AppDaemon add-on config, depending on the method you used for installing.

https://github.com/efficiosoft/hass-apps/archive/master.zip

Then, this simplified postprocessor should do the trick:

- x: "Postprocess(lambda result: round(result, 1))"

[asjmcguire]

Confirmed working using the much shorter Postprocess line.

2020-07-12 11:58:14.534722 INFO schedy_heating: --- [R:House]         ������ => 19.4
2020-07-12 11:58:14.537983 INFO schedy_heating: --- [R:House] Applying postprocessors.
2020-07-12 11:58:14.541359 INFO schedy_heating: --- [R:House] + Add(-0.1��)
2020-07-12 11:58:14.544766 INFO schedy_heating: --- [R:House] = 19.299999999999997��
2020-07-12 11:58:14.548137 INFO schedy_heating: --- [R:House] + Postprocess(<function <lambda> at 0x7fcb2e713ee0>)
2020-07-12 11:58:14.551472 INFO schedy_heating: --- [R:House] = 19.3��
2020-07-12 11:58:14.554869 INFO schedy_heating: --- [R:House] Final result: 19.3��, markers: set()
2020-07-12 11:58:14.558081 INFO schedy_heating: --- [R:House] Result didn't change, not setting it again.

[bob1de]

Ok, then this will be included in the next release.

Thanks for sorting it out with me.