Most sensors unavailable

[DanDon]

Not sure if the dependencies need bumping or if there is currently an issue with the Glow API but most of the sensors are unavailable most of the time recently.

All seemed to be working fine before but the last week sees very intermittent availability of the sensors.

This also seems to be the error that is happening more regularly than it did before a week ago....

[CoreyD97]

Just found the same problem. I can't see any changes to the API on the Glowmarkt website, hopefully just an outage on the API?

[ColinRobbins]

Working OK for me. I suggest it the “usual” glow server instability.

[harveyorg]

I'm getting the same issue.

If the API server is known to be unstable, should the data collectors be made more robust and keep trying until they get a response? If the data is only changing every 30 mins then the value isn't actually changing.

Once one value is missed then the next value plotted is also incorrect because it's the delta from last good reading.

e.g. in the graph the 6:00 reading is double what it really was.

Here's the data that Bright app shows

But I also see missing data in Bright app - here's the data before

I can see Loop app and Octopus have data - but they're dallied by 24hrs.

Do these issues go away if you buy the hardware?

[ColinRobbins]

With the way HA works, there is nothing more you can do on the client side. Once data is missed, when you eventually get a new reading it will show as spike.

Sorry, I don’t know about the hardware, I’ve learnt to live with the free data - accepting it has bugs and there in nothing more HA can do, the issues are server capacity related.

@HandyHat did hint at a complete re-write of the integration to reduce server load, with might make a small improvement in reliability.

[antonoal]

Isn’t it possible with HA not to render sensor unavailable when a reading is missed but rather stay as it used to be before a failed reading?

[ben-willmore]

Isn’t it possible with HA not to render sensor unavailable when a reading is missed but rather stay as it used to be before a failed reading?

If you want to track the last valid values, you can make template sensors like these:

template:
  - sensor:
      - name: "Electric Consumption (Today, Valid)"
        icon: mdi:flash
        unit_of_measurement: "kWh"
        device_class: energy
        state_class: total_increasing
        state: >-
          {% if states('sensor.electric_consumption_today') == 'unavailable' %}
            {{ states('sensor.electric_consumption_today_valid') }}
          {% else %}
            {{ states('sensor.electric_consumption_today') }}
          {% endif %}
  - sensor:
      - name: "Gas Consumption (Today, Valid)"
        icon: mdi:fire
        unit_of_measurement: "kWh"
        device_class: energy
        state_class: total_increasing
        state: >-
          {% if states('sensor.gas_consumption_today') == 'unavailable' %}
            {{ states('sensor.gas_consumption_today_valid') }}
          {% else %}
            {{ states('sensor.gas_consumption_today') }}
          {% endif %}

[harveyorg]

I'm sure this has been talked about before - but I'm just getting started in HA.

The issue here isn't the "unavailable" state itself, because even if you ignore it and set it to the last reading, it will still appear as an empty column on the energy dashboard - which tracks the delta from the last value.

If we accept that the GLOW API isn't stable (in my case it fails about 10%) then the other approach is to make the integration more robust and so when it gets 200 response off the API it scans back over the previous missing readings and gets them too.

At the moment the current usage params are from midnight 00:00:00 to end of day 23:59:59, but it could be modified to pick up previous values e.g. 00:00:00 to 13:59:59 in order to pick up a missed value for 13:00

My python is a bit rusty, so will take me some time to do this, but if anyone else is up to it, it doesn't seem like a major change.

Another factor might be that all HA integrations are hitting the API at the same time - I don't see where that's done though.

[ColinRobbins]

The challenge with the HA integration model, is you cannot add historic data. Once a reading is missed, you can't add it later. All you can do is give a current total meter reading when the API is up. HA will take this, and add it into the current hour. You can't tell HA to "back date" some of it. No data is lost - its just accounted for in the wrong hour. So even if you read the missed intermediate value from HA- you would not be able to get HA to do anything sensible with it.

Another suggestion higher up is to not report "unavailable" and give the value from the previous sucessful attempt. Again this will not help, as the delta will be zero, until you get a new reading.

Yes a factor is all HA integration hitting the API at the same time. That is a Glow issue. The Glow API updates the data on the hour, and (if you are lucky) 30 minutes past. The HA integration will attempt a read between 00 and 05 minutes, to provide some randomisation.

There are improvements that the HA integration can make (with a breaking change) to reduce the number and types of sensors, so reduce the number of calls on the Glow API. I.e., reduce flexibilty to gain reliabilty. @HandyHat was looking at this, but is a lot of work.

[Eddie46848]

I've disabled the sensors I don't use, so I'm left with just day use and day cost. I've not had a drop out for 2 days. Might be worth a try.

[harveyorg]

Sounds good. Even if it only reduces it it’s worth it. Can you say what you did?

[HandyHat]

This should no longer be an issue as of v1.0.0. If this issue persists, please say!

[harveyorg]

@HandyHat yes, fixed. removed old version and reinstalled v1.0 through HACS and no error. thanks. BTW - I opened a new discussions on some anomalies I'm seeing in V1, which I think is the underlying data and not the integration.