[BUG] Battery operated devices appears as "unavailable after some time

galex80 commented 4 years ago

Environment

ShellyForHASS release with the issue: 0.1.9
Last working ShellyForHASS release (if known): I guess ~ 0.1.6
Home Assistant Core release with the issue: 0.112 - 0.113
Operating environment (Home Assistant/Supervised/Docker/venv): HASSIO - Home Assistant

Describe the bug

Sometimes it happens, that one or multiple Door / Window Sensors appear as "unavailable". When i check the details, one sensor change the status from "Closed" to "Unavailabe" 10 hours ago, another one 3 hours ago. I never saw this before, only when the batteries are empty and the device is dead. If i force the sensor waking up (e.g. open / closed the window/door) than its "alive" again. This happens not only for specific sensors, it happens randomly for mostly every of my D/W sensor at no specific time

Steps to Reproduce

Unfortunately there is no way to reproduce, it simply happens randomly

Expected behavior

It should stay with the last status. Maybe only mark them as "unavailable" when it doesn't update for over 72 hours

Screenshots

Traceback/Error logs

Additional context

chemelli74 commented 4 years ago

Same goes for Shelly Flood.

Simone

felixpackard commented 4 years ago

I am having the same issue with a Shelly H&T sensor - worked perfectly for a few days, now temperature, humidity and battery all show as unavailable.

sparcopt commented 4 years ago

Same here with the H&T sensor. Started to show as unavailable for some days. Then it started showing values but they don't match the values from the Shelly application. It also displays this message quite often:

"This entity is currently unavailable and is an orphan to a removed, changed or dysfunctional integration or device. If the entity is no longer in use, you can clean it up by removing it."

sparcopt commented 4 years ago

I eventually pressed the H&T's button and restarted HA to let if find the H&T again. Values were back but after 2-3 days it started to show unavailable again.

Just sharing my experience as it might be usefull for debugging the problem.

chemelli74 commented 4 years ago

Screenshot of current situation:

Simone

TheGroundZero commented 4 years ago

I believe this is linked to my issues with the SW2 (#421) and H&T (#422).
Still present in 0.2.0

hakana commented 4 years ago

Yes, you can try the igmp_fix setting and see if that help

Nick3C commented 3 years ago

I have also experienced this issue with four H&T sensors.

After initial set-up, the H&Ts work in HA for a short period (a few hours) and then either stop updating their values randomly, or (especially following a HA restart) simply show as "Unavailable" or "Unknown". The only way to get them working again seems to be by manually turning on the sensor using the switch (and/or setting them up again).

Based on another now-closed issue thread (here: https://github.com/StyraHem/ShellyForHASS/issues/74), I pulled the data directly from Shelly's cloud:

- platform:
 rest
  name: sensor.shellycloud_living_room_temperature
  unit_of_measurement: "°C"
  device_class: temperature
  resource: https://shelly-XX-XX.shelly.cloud/device/status
  method: POST
  payload: auth_key=XXXX&id=XXXX
  scan_interval: 60
  headers:
    User-Agent: Home Assistant
    Content-Type: application/x-www-form-urlencoded
  value_template: 
"{{ value_json.data.device_status.tmp.value }}"

This works but, in the interest of trying to get local data, I also played around with a template to select the local data if available and if not use the cloud data (based on the linked post above as well, with some tweaks):

- platform: template
  sensors:
    living_room_temperature: 
      friendly_name: "Living room temperature"
      unit_of_measurement: '°C'
      value_template: >-
        {% if states('sensor.shellyht_living_room_temperature') == 'unavailable' %} 
          {{ states('sensor.sensor_shellycloud_living_room_temperature') }}
        {% elif states('sensor.shellyht_living_room_temperature') == 'unknown' %} 
          {{ states('sensor.sensor_shellycloud_living_room_temperature') }}
        {% else %}
          {{states('sensor.shellyht_living_room_temperature')}}
        {% endif %}

This sort of works (as it resolves the issue with unknown and unavailable local sensors) however, it doesn't resolve the issues with outdated local values, so in the end I gave up and decided to rely on the cloud data.

Appreciate that this could be done using MQTT (but then you would lose the Shelly cloud / app access). Seems a lot of people have this issue.

rousveiga commented 3 years ago

Same here, I've seen this happen for all my battery-operated Shellies (DW2, H&T and Motion). igmp_fix doesn't seem to solve the issue.

I have version 0.2.2 of ShellyForHass and version 2021.3.4 of Home Assistant.

Edit 05/04/21: I noticed that it is mostly the auxiliary sensors that go unavailable. For example, the DW2 door sensor does display the last "open" or "closed" state, but the illuminance and temperature sensors display the value "unavailable".

rousveiga commented 3 years ago

I've been experimenting with the unavailable_after_sec setting, set it to 10 minutes for all of my battery Shellies.

The Shelly H&T seems to work fine, and it worked fine before changing unavailable_after_sec. It delivers updates anywhere between every 15 minutes and every couple of hours. Never goes unavailable, despite the update frequency exceeding the unavailable_after_sec timeout. This feels like the correct way to behave. It hasn't always worked like this, but I haven't done anything special to the HA or device configurations to make it that way.

The Shelly Motion and Shelly DW2, however, seem to go unavailable when there's long periods of time without the need for main sensor updates (if the door is kept closed, or no motion is detected, for example at night). There is this weird occurrence where, if I trigger a main sensor update, the devices will keep sending updates for the auxiliary sensors for a few hours (and the 10 minute timeout seems to be enough), but they stop after a while - leading to artifacts like this in graphs:

This is the temperature sensor for a Shelly DW2. The Shelly Motion does the same thing. Happens with both main and auxiliary sensors.

Hopefully this info can help in diagnosing the issue.

ricarva commented 3 years ago

@rousveiga @Nick3C @TheGroundZero @deadpixxl

What is your WiFi like?

As already documented, some WiFi setups have trouble with the CoAP messages (network multicast), and don't propagate them.

Main offenders seem to be systems with any kind of wireless backhaul: most WiFi repeaters, modern wireless-mesh solutions, etc.

I've experienced similar problems in the past and eventually was able to figure out it was a problem with my mesh/repeater.

scooper1 commented 3 years ago

Try to check the mdns is working correctly on openwrt I had to load some extra modules to get multi cast to work correctly. Search Google for how to test multicast on your system. Remember to check on your machine that is running home assistant as can be a internal problem very hard to give instructions on how to do as some many different ways of doing things

hakana commented 3 years ago

Yes sounds like problem with CoAP.

Try to config as unicase or use mqtt support in ShellyForHass.

rousveiga commented 3 years ago

I see. My network setup does involve a few different access points, so it might be that. I'll configure as unicast and see if that solves the issue. Thank you!

rousveiga commented 3 years ago

I configured as unicast, and the behavior hasn't improved (for instance, you need to shake the Shelly Motion, therefore triggering the vibration sensor, for it to start sending updates).

Feels more like a Shelly problem than a ShellyForHass problem, however. I think I need to drop by a few Shelly forums and read some documentation to really understand what the device is doing and how does it decide it's time to wake up and send updates.

ricarva commented 3 years ago

@rousveiga I hope you can figure it out.

One thing though: whatever you're doing in the network to configure things as unicast, I'm not sure that will improve things.

My understanding is that for Shelly CoAP to function properly, it requires multicast to be working with no hiccups. Configuring the network to expressly use only unicast might actually compound on any issues that were already present.

rousveiga commented 3 years ago

Ah, I haven't done anything special to the network configuration. What I meant by "configuring as unicast" was that I set the CoIoT peer in the Shelly configuration:

Sorry if that was the wrong wording. I'm somewhat of a noob at network administration 🙂

ricarva commented 3 years ago

@rousveiga

No, actually I'm the one still learning things: I was oblivious to that setting in the Shellys! Thanks for the share, and I again hope you can figure it out.

rousveiga commented 3 years ago

Thank you! 😀

rousveiga commented 3 years ago

I had overlooked one very useful feature in the Shelly DW2. This was disabled by default:

I enabled it, and ever since my Shelly DW2 has been waking up throughout the day to report illuminance and temperature values. This is the illuminance history:

I'm very happy with these results, hopefully it will last.

As for the Shelly Motion, it has a bug that causes it to never wake up from low power mode if it doesn't detect any motion for a long time. It has been reported and the devs are working on it: https://www.shelly-support.eu/forum/index.php?thread/8921-device-is-offline/ https://www.shelly-support.eu/forum/index.php?thread/8905-wifi-re-connect/

StyraHem / ShellyForHASS