option to force sensors not go unavailable via api

[Bascht74]

Describe the problem you have/What new integration you would like Hi, I am using an api connection right now and I am facing some sensors go unavailable in HA because of deep sleep and/or not 100% reliable connection. I don't want those sensors to go unavailable, at least for for some hours. I don't want those sensors to go unavailable, at least for for some hours.

right now it looks like this in HA:

I would like something similar the option of sensors for mqtt expire_after:

e.g.:

api:
  reboot_timeout: 240 min
  device_unavailable_timeout: 180 min
  encryption: 
    key: !secret encryption_key

this tells the HA integration not to set entities unavailable right away, only after the time specified. Maybe possible in the entitiy config instead of a global configuration:

  - platform: template
    name: "${upper_devicename} Test"
    id: Test
    unavailable_timeout: 180 min

Please describe your use case for this integration and alternatives you've tried: The only available option is mqtt and the use of the last will message. But I would more like to use api instead of mqtt.

Additional context

[nagyrobi]

You could use HA's template sensor for that.

[alco28]

yes I also like to see this native in esphome. because i don't like to create copy sensors in HA template for just this reason

[nagyrobi]

Related to https://github.com/esphome/feature-requests/issues/1568, but the other way around.

Also partly related to https://github.com/esphome/feature-requests/issues/1492.

[Bascht74]

@alco28 Thx for pointing that out. I think, too the API should be able to produce the same result as MQTT (I switched to that because of this as I don't want to copy sensors in HA, works fine so far).

[Bascht74]

Also partly related to #1492.

I don't think so in this case.

[crlogic]

@alco28 Thx for pointing that out. I think, too the API should be able to produce the same result as MQTT (I switched to that because of this as I don't want to copy sensors in HA, works fine so far).

I don't think you need to copy any sensors in HA.

What about having the ESPHome sensor publish to an HA input_number. The input_number would never go offline as it is an HA helper.

Might be a workaround until/if the desired behavior is added.

[nagyrobi]

I don't want those sensors to go unavailable, at least for for some hours.

A bit offtopic, but why do you want that? You'd fool your Home Assistant system thinking that the sensors value hasn't changed, which will lead sooner or later to other problems (eg. if you control a heating system and your sensor stays the same infinitely due to unreliable connection, you'll end up overheating or not turning on when required, causing a multitude of other problems).

IMHO it's always better to have the availability state correctly passed to HA, and then handle the situation smartly there (eg. read from another sensor, use values from yesterday, use some default value, or even do as you wanted, keep it unchanged; + send you a notification to go and fix the sensor).

But this is just my humble opinion.

PS: BLE sensors behave by default exactly how you want, extra stuff is needed to make them unavailable when they go offline: https://github.com/esphome/feature-requests/issues/1687

[Bascht74]

@nagyrobi Sometimes the data is more important than the 15s availability checking of the device. So I want the check on the data (mqtt: expire_after) and not the device itself (like your example with the BLE sensors).

As I said, I don't want to keep the data forever and pretent that it is current. But I don't want them to go offline for a specific timeout value via API, similar to the mqtt expire_after and keepalive parameters.

I would see something like:

api:
  keepalive (optional): 15s # sets the device to unavailable after the time specified, if the device is not connected via API. Defaults to 15 seconds

sensor:
  - platform: whatever
     expire_after (optional): 600s # sets the entity to unknown after the time specified, if a sensor is not sending any data (the same data = no update would be OK, too)

This way, you can use it not only with mqtt but with API and you can controll the unavailable state of the device and the unkown state of the entities the way you like

[breti]

I have noticed that entities do not change to "unavailable" when deep sleep is configured.

In the ESPhome integration for Home Assistant I also found the corresponding code:

https://github.com/home-assistant/core/blob/dev/homeassistant/components/esphome/__init__.py

    @property
    def available(self) -> bool:
        """Return if the entity is available."""
        device = self._device_info

        if device.has_deep_sleep:
            # During deep sleep the ESP will not be connectable (by design)
            # For these cases, show it as available
            return True

        return self._entry_data.available

[luixal]

Hi

I was just looking for this feature.

This is my use case:

I have an esphome sensor for measuring the level of a ph- tank in my pool system. As this tank's level can only change while the filter is working, the esphome sensor is only powered while the filter is (energy is precious). When the filter and sensor power goes off, the sensor becomes unavailable in HA instead of keeping it's value, which would be the proper thing.

Being able to manage/force this would be great.

Thanks!

[ssieb]

@luixal that's a different issue. Is it unavailable or nan? If you don't want the nan, you can filter it out.

[breti]

I have an esphome sensor for measuring the level of a ph- tank in my pool system. As this tank's level can only change while the filter is working, the esphome sensor is only powered while the filter is (energy is precious). When the filter and sensor power goes off, the sensor becomes unavailable in HA instead of keeping it's value, which would be the proper thing.

@luixal Just add

deep_sleep:
  id: id_deep_sleep

to the YAML config of the ESPhome device. I think this will keep the device from being unavailable.

[luixal]

Hi,

Adding the deep_sleep config @breti suggested didn't work, device kept going unavailable.

I already had a filter_out: nan in my config @ssieb , but your comment took me to reading again the filters section and notice the timeout one:

After the first value has been sent, if no subsequent value is published within the specified time period, send NaN. Especially useful when data is derived from some other communication channel, e.g. a serial port, which can potentially be interrupted.

According to that I added another filter:

filters:
  - timeout: 1d

Didn't work. Tried this also:

filters:
  - filter_out: nan
  - timeout: 1d

Didn't work either.

This is totally crazy, any ideas on how to accomplish this?

Thanks!

[breti]

Did you remove the device from HA and add it again to HA after adding deep_sleep config?

[luixal]

Not really, but I don't thin that's needed. If I add a deep_sleep config (i.e: run 15mins, sleep for 10mins) it's doesn't go unavailable while sleeping.

This is, config changes in ESPHome doesn't seem to need anything in HA.

[breti]

This is a different case, I think. HA is actively informed about the upcoming sleep.

[luixal]

Ok, will try it.

It seems I could force something like this if using mqtt (will_message), but can't find the same option when using API.

[luixal]

Just tried it by creating a new device/sensor abd still going unavailable.

[breti]

Hm ok :-(

[ssieb]

@luixal the default value for the timeout: filter is to send nan, so it's not so useful as is. :-) However, I made a change to it that lets you specify the value to send. It will be available in the next release or it's in dev and beta now if you want to try. Or you can use the PR as an external component. But this was a side-track from the original request as I mentioned way back...

[luixal]

I don't get it @ssieb , how would sending a fixed value help?

If the value sent was the last value from sensor it would be ok, but no clue how to use a specific value (althoug you filter it out and that would work).

Thanks!

PS: IMHO, issue title and description matches what I need and the solution proposed by the author would solve my problem too.

[ssieb]

The original issue is due to deep sleep or unreliable wifi. Deep sleep is already supported, so the only request here is to handle intermittent outages of wifi. For your case, it was unclear what you were asking. I thought it was just the sensor, but now I'm guessing that you're actually powering off the ESP as well. If you have some control over when it will power off, then setup deep sleep and trigger that before the power goes off. Then HA will assume it's still in deep sleep and won't make it go unavailable.

[luixal]

Oh! What a missunderstanding! Yes, I'm powering the esp board off.

I could hanlde it with deep sleep as you say, updating the sensor when I change the run schedule in HA, but the requested feature in this issue (the expire_after field) would perfecly solve my issue without having to handle this in two different things, as I could prevent the sensor to go unavailable for 24 hours :)

[ssieb]

With deep sleep triggered, it shouldn't ever go unavailable.

[luixal]

True, that works.

I'm looking for the propper solution, which the proposed in this issue by the author.

I have 6 sensors like this one that run on different schedules. This is, HA turns on 6 switches at different times, those switches turn equipments and esp boards on. I change this schedules several times per year (some more often than others). Using deep sleep means that everytime I change a schedule in HA, I have to edit and update the esp too.

With the feature proposed here, I wont have to do that ever.

[ssieb]

Well, I'm giving you a workaround because there's no guarantee that your "proper" solution would ever be implemented. You're not really using deep sleep, so the schedule is irrelevant. The deep sleep is just to make HA think it's sleeping. The off time can be as long as you want, you don't have to tell it. HA doesn't know how long the deep sleep time is set for.

[luixal]

So, if I've understood it right:

My pool pump works from 12 to 18h. The esp is powered while the pump is powered. If I set an deep_sleep with a run time of 12h and an sleep time of 1h, it would work the powered hours and never go unavailable, right?

If it works that way, true, I wont need any proper solution for this. I woudl make a PR for the docs to reflect this use case and explain this way of using it :)

[ssieb]

You don't need to specify the run time or the sleep time. If you don't, it will run until you tell it to sleep and then it will sleep until you reboot it. (You're powering it off anyway.)

[luixal]

If it's powered off before going into sleep it still becomes unavailable in HA?

[ssieb]

I'm not sure. You'll have to test that.

[luixal]

Just tested. No, device goes unavailable. I guess it is mandatory that it goes to sleep before it is powered off so... I have the same problem :(

[ssieb]

You don't have any warning before the power goes off? I guess if you don't have that, there's no work around for now.

[luixal]

Schedule is managed by HA automations, I could add something to tell the esp to enter sleep mode I guess... I've never played around with that, will give it a try.

Do you have some example related to that?

Thanks!

[ssieb]

You could use a service call or a template button.

[luixal]

Ok, just found it. I guess I can define a service on the esphome device to get into sleep and call it from the automation running the schedules.

It would also cover manual start/stops :)

Will try it out soon.

Thanks!

[luixal]

Just an update for anyone getting here.

You have to be cautious with the sleeping time you add in the service, mine is configured like this:

api:
  encryption:
    key: "..."
  services:
    - service: sleep
      then:
        - deep_sleep.enter:
            sleep_duration: 5s

I first try to set a quite higher sleep_duration, just for the device to have enough time for powering off while the esp is sleeping but found out that, when powered on, it would sleep for the rest of the time. This is, if you set like 1h and then power off the device, when the device is powered on, it will still sleep that hour. Or that's what it seemed from my tests.

In my automation, I set a sequence for powering off that first tels the esp device to go to sleep, like this:

- service: esphome.ph_level_sensor_sleep
  continue_on_error: true
  data: {}

and then it powers off the device itself.

I then updated my lovelace card so, instead of toggling the switch, it calls this automation for powering off (toggling actually, but that's too much unrelated code to put here).

One caveat of this, and that's why I set it to 5secs, once the switch is on, if I tap the card, it's needed that the esp device has finalized sleeping and it has connected to the wifi. Bad thing if you just want to switch it on and off quickly. I use the continue_on_error in the automation just in case something went wrong.

I can't find any info about a sensor or something similar to know if an esp device is connected, that could be really handy :)

Finally, I still think the expire_after config option would be really useful and would allow me to solve this without all this stuff around, will be great to see it coming to esphome :)

Thanks!

[Jeppedy]

Unless I'm missing something, DeepSleep is still causing 'Unavailable' as soon as it goes to sleep. In my ESPHome device, I'm referencing a helper state in HA, if that makes a difference.

[luixal]

Hi @Jeppedy, not in my case. When device goes into deep sleep it doesn't go into unavailable state in HA. This only happens to me when device is powered off BEFORE entering deep sleep.

Would be totally logical that state and value were different things (as they, conceptually, are) but that's more a HA issue.

[Jeppedy]

Odd. Here's my experience:

As soon as the device goes to DeepSleep, it flips to Unknown. Pretty straightforward. Am I doing something wrong?

substitutions:
  device_name: househumidity-esp32s
  friendly_name: househumidity-esp32s
  deepsleep_time: 30s
  update_interval: 5min
  sleep_check_interval: 5s       # As soon as the switch is toggled, we want to go to sleep.
  initial_sleep_check_delay: 30s  # Allows API to connect and give sleep boolean to us

esphome:
  name: ${device_name}
  friendly_name: ${friendly_name}

  on_boot:
    then:
      #- logger.log: 'on_boot - Before initial delay'
      - delay: ${initial_sleep_check_delay}
      #- logger.log: 'on_boot - executing script'
      - script.execute: consider_deep_sleep

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "xxxxx"

#ota:
#  password: "xxxxx"

#captive_portal:

wifi:
  power_save_mode: none

  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
#  ap:
#    ssid: ${device_name}
#    password: "xxxxx"

sensor:
  - platform: dht
    pin: 16
    temperature:
      name: "Temperature"
    humidity:
      name: "Humidity"
    update_interval: ${update_interval}
    model: DHT11

## Deep Sleep

deep_sleep:
  id: deep_sleep_control
  sleep_duration: ${deepsleep_time}

# Will only pick up a value if this device is configured in Home Assistant > Integrations
# If the device isn't configured, or Home Assistant is offline, it'll default to false
binary_sensor:
  - platform: homeassistant
    id: prevent_deep_sleep
    entity_id: input_boolean.prevent_deep_sleep

script:
  - id: consider_deep_sleep
    mode: queued
    then:
#      - logger.log: 'Starting Script iteration'
      - delay: ${sleep_check_interval}
#      - logger.log: 'After Script delay'
      - if:
          condition:
            - api.connected:
          then:
            - delay: 5s
            - if:
                condition:
                  - binary_sensor.is_off: prevent_deep_sleep
                then:
                  - logger.log: 'No prevent_deep_sleep, so nappy nap time'
                  - deep_sleep.enter: deep_sleep_control
                else:
                  - logger.log: 'Staying awake; prevent_deep_sleep set.'
          else:
            - logger.log: 'Skipping sleep - no API connection.'
      - script.execute: consider_deep_sleep

[luixal]

Have you tried setting run_duration along with sleep_duration?

I have that in my config.

[mingshenggan]

Did you remove the device from HA and add it again to HA after adding deep_sleep config?

Recommend above answer - remove and readd if you haven't tried. It is also listed in EspHome FAQ.