Ruuvi Gateway integration not working

[Linqman]

The problem

After updating Home assistant to the newest version, my Ruuvitag Gateway was not immediately auto-discovered, but I was able to add it manually. However, no devices were found, and the logs said the integration couldn't connect. I deleted the integration and restarted HA, and now my ruuvi Gateway was auto-discovered, and I added it again without problems. It still doesn't report devices and the logs says "Bluetooth scanner has gone quiet for 90s, check logs on the scanner device for more information".

I have up until now been publishing the data from the gateway with MQTT, which I now disabled to test the new integration and to keep the logs smaller for testing.

What version of Home Assistant Core has the issue?

2023.2.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Ruuvi Gateway

Link to integration documentation on our website

https://www.home-assistant.io/integrations/ruuvi_gateway

Diagnostics information

home-assistant_ruuvi_gateway_2023-02-02T09-17-27.088Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

If I manually run the command "curl -H "Authorization: Bearer [correct bearer token]" http://192.168.XXX.XXX/history", I get proper history data from the Ruuvi gateway.

[home-assistant[bot]]

Hey there @akx, mind taking a look at this issue as it has been labeled with an integration (ruuvi_gateway) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `ruuvi_gateway` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign ruuvi_gateway` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

ruuvi_gateway documentation ruuvi_gateway source _{^{(message by IssueLinks)}}

[sj3fk3]

I've got the exact same issue, so it's not a n=1

[Jonssii]

Same here, Home Assistant finds the Gateway but none of my Ruuvitags show up. I use MQTT to connect the Ruuvitags, tried disabling MQTT integration to no effect.

Ubuntu + HA Docker

[plepola]

Same here.

Raspberry Pi + HomeAssistant

[jarilehtinen]

Same here. Home Assistant auto-discovered the gateway, added the bearer token, but nothing happens after that.

Generic x86-64 on Intel NUC.

[akx]

Author of the integration here. Sorry for the trouble.

That's curious – it certainly worked fine when I implemented it, but now that I tried with the release version of HA 2023.2, it's not doing much at all.

Um, @bdraco, since you know more about the Bluetooth systems in HA, can you think of a change that could have caused this, or should I just dig deeper myself?

[akx]

Never mind, I think I found the bug.

As always, time comparisons are hard... ⏳

[Misentry]

Still seems to be broken, I verified that bearer token works in curl:

➜ automation curl -H "Authorization: Bearer " http:///history { "data": { "coordinates": "", "timestamp": "1675592599",

With Debug logging in HA; only entry in log regarding this integration is: 2023-02-05 12:19:54.602 DEBUG (MainThread) [homeassistant.components.ruuvi_gateway.bluetooth] Ruuvi Gateway ]: Connecting scanner

[Misentry]

I changed token, and left it unchanged in integration, some more logs:

httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'http:///history' For more information check: https://httpstatuses.com/401

So it seems to connect with current token, but does not get/parse anything from /history?

[plepola]

It works after 2023.2.2 update! For some reason, every time after booting, the Gateway is found as a new device and needs to be defined again.

[Misentry]

Are you sure it actually works? Because I have 2023.2.2, and still does not actually find any of the tags, and it's a sign that it does not work if it get's discovered again after every reboot

[Jonssii]

Definitely works for me, just set up everything. Gateway not discovered again after rebooting.

[Misentry]

Then I wonder if it's conflict with Ruuvi BLE and/or Bluetooth BLE integration.

If you look for Ruuvitag in Devices view, what's the integration for them? I tried removing one, and it's get discoverd by Ruuvitag BLE intergration

[plepola]

Are you sure it actually works? Because I have 2023.2.2, and still does not actually find any of the tags, and it's a sign that it does not work if it get's discovered again after every reboot

It is odd but true that Gateway does not find new devices, but sensors are found as separate BLE devices. I can't be sure if the connection is through the Gateway, but the sensors have been too far for a direct Bluetooth connection in the past. Now the signal seems to be stable.

[plepola]

Definitely works for me, just set up everything. Gateway not discovered again after rebooting.

Are your sensors separated BLE devices or are they visible under Gateway integration?

[Misentry]

I wish integration documentation would state if RuuviTags are actually listed under devices of the same integration, or are they listed as Ruuvitag BLE?

[Jonssii]

Are your sensors separated BLE devices or are they visible under Gateway integration?

They are separated BLE devices under ruuvitag BLE. And it is using the Gateway, if I disconnect the Gateway power supply they sensors stop updating. There is nothing listed under the Gateway integration.

[Misentry]

Actually when I disconnect Bluetooth adapter, they are still updated.

So it works, but is rather confusing with lag of debug log updates other than: : Connecting scanner and devices appearing under RuuviTag BLE.

I still have the problem with being discoverd after every boot, even if previosly configured one seems to work now.

[plepola]

I still have the problem with being discoverd after every boot, even if previosly configured one seems to work now.

Same here. It is really annoying.

[akx]

Then I wonder if it's conflict with Ruuvi BLE and/or Bluetooth BLE integration.

I don't think that's the case – the Ruuvi Gateway integration essentially just works as a remote bluetooth adapter (like you can enable ESPhome or Shelly v2 devices to do), and relays Bluetooth advertisements for the rest of HA (e.g. the Ruuvi BLE integration, also authored by yours truly).

For some reason, every time after booting, the Gateway is found as a new device and needs to be defined again.

@bdraco Can you think of some obvious reason for this? (I'm thinking something about unique IDs..?) I'm assuming the Gateway's IP isn't changing every boot of @plepola's HA system or anything...

[Linqman]

The gateway integration is now partially working for me. HA discovered 3 RuuviTags, which are working with the gateway, however, I have 6 RuuviTags connected to it, so only half are discovered. I also have 4 Xiaomi temperature sensors connected to HA, and they now also seem to get data through the gateway, as they stopped updating when I disconnected the gateway.

For testing purposes, I removed a Custom BLE integration, but that did not seem to have any effect. I also disabled the Bluetooth integration for the onboard Bluetooth adapter.

I also have the Ruuvi Gateway autodiscovered each time I restart HA.

[plepola]

@bdraco Can you think of some obvious reason for this? (I'm thinking something about unique IDs..?) I'm assuming the Gateway's IP isn't changing every boot of @plepola's HA system or anything...

Yes, the IP stays the same. It is locked in the router settings.

[bdraco]

@bdraco Can you think of some obvious reason for this? (I'm thinking something about unique IDs..?) I'm assuming the Gateway's IP isn't changing every boot of @plepola's HA system or anything...

You are using the mac as the unique id (this is a good thing) so it shouldn't matter if the ip changes.

It looks like its set correctly in both places.

https://github.com/home-assistant/core/blob/dev/homeassistant/components/ruuvi_gateway/config_flow.py#L86 https://github.com/home-assistant/core/blob/dev/homeassistant/components/ruuvi_gateway/config_flow.py#L48

@plepola Can you try setting it up again and than post the entries you see in .storage/core.config_entries. Maybe that will reveal what the issue is.

[plepola]

@plepola Can you try setting it up again and than post the entries you see in .storage/core.config_entries. Maybe that will reveal what the issue is.

Here they are. There are no differences between them, only the entity_id differs. XXX are censored.

Before reinstall { "entry_id": "db3a48a935318cf13da4dce1327a9246", "version": 1, "domain": "ruuvi_gateway", "title": "Ruuvi Gateway A9:28", "data": { "token": "XXXXXX", "host": "192.XXX.XX.XXX" }, "options": {}, "pref_disable_new_entities": false, "pref_disable_polling": false, "source": "dhcp", "unique_id": "XX:XX:XX:XX:a9:28", "disabled_by": null }

After reinstall { "entry_id": "dbd4fadd49cc3180f6ea837525a67224", "version": 1, "domain": "ruuvi_gateway", "title": "Ruuvi Gateway A9:28", "data": { "token": "XXXXXX", "host": "192.XXX.XX.XXX" }, "options": {}, "pref_disable_new_entities": false, "pref_disable_polling": false, "source": "dhcp", "unique_id": "XX:XX:XX:XX:a9:28", "disabled_by": null } ] }

[plepola]

Is the reason here: "pref_disable_new_entities": false,

[bdraco]

I'm missing something but both entries have a source of dhcp so it shouldn't have been possible to get past this line: https://github.com/home-assistant/core/blob/dev/homeassistant/components/ruuvi_gateway/config_flow.py#L87

[plepola]

After the Home Assistant 2023.2.3 update bug is still alive. After booting, the Gateway is always found as a new device.

[Linqman]

My Gateway stopped appearing as a new device while I was still on the 2023.2.2 version. The thing I did that might have affected this is I removed my normal Bluetooth integration as well as my Bluetooth Low Energy Monitor integration, and then after a reboot I added them again. Now I no longer get the Gateway found as s new device on reboot.

[plepola]

My Gateway stopped appearing as a new device while I was still on the 2023.2.2 version. The thing I did that might have affected this is I removed my normal Bluetooth integration as well as my Bluetooth Low Energy Monitor integration, and then after a reboot I added them again. Now I no longer get the Gateway found as s new device on reboot.

I tried to remove Bluetooth and iBeacon Tracker but it had no effect, the gateway still appears as a new device after booting.

Then I skipped the newly founded Gateway installation and hid it. After booting, no more new devices, and the old Gateway integration still works. It doesn't sound like a sophisticated solution in the long run, but it works for me.

[mrl0leak]

I'm running HA in Proxmox VM, so I don't have Bluetooth available at all. Installed VM to Proxmox with that installation script found from: https://community.home-assistant.io/t/installing-home-assistant-os-using-proxmox-7/201835

Installed yesterday RuuviGateway integration and it found 6 (!?!) of my 9 RuuviTAGS. Configured 2 of those and was amazed how well it worked. Gateway integration seemed to work flawlessly.

Today I was planning to configure rest of my TAGs and find out reason why 3 TAGs was missing. Now those 2 TAG show "unavailable" and other TAGs are not showing in Devicelist at all. It looks to me that these TAGs are BLE devices and because I don't have Bluetooth they don't show anything. Gateway integration seems completely mute.

What should I try next? Should I just wait integration gets fixed or do I need to uninstall something?

[Misentry]

One curious thing to add is that Ruuvi Gateway is not listed in Devices, even when it's seems to be working

[mrl0leak]

Is it possible that problem is at Gateway end? Tried to remove/reinstall Gateway integration from HA. Same configuration was not working anymore, Gateway token was changed. configured gateway with new token and it seems to work again. Let's see what happens in few hours.

How is it possible that token changes in gateway?

[Linqman]

It might also be that the reason my gateway stopped showing up as a new integration was that I enabled and configured the 3 Ruuvitag BLE devices that I thought was missing. I had previously disabled them as using them through my onboard bluetooth adapter was not reliable.

[plepola]

Is it possible that problem is at Gateway end?

The bug has not disappeared. However, it started to work for me when I disabled the reappeared device. If I enable it, the new device always appears after boot.

[plepola]

Home Assistant found a new Gateway device this morning. The old device still shows up as an integration, so I bypassed and disabled the new one. The old Gateway integration and tags work.

[mrl0leak]

OK, got it now.

I think HA integration is OK. Problem is RuuviGateway itself.

My ruuvi-readings were working about a week in HA. This morning I noticed everything was "unavailable" again. So I opened my RuuviGateway configuration and yes, "Enable bearer authentication" was unchecked. After re-activation new token was generated by Gateway. But I overwrote it by the one I used to configure HA-integration week ago. And you guessed it, integration started to work again.

At least for me only problem seems to be RuuviGateway, which switches bearer authentication off by itself. I hope in future firmware updates Gateway stops this self-configuration, and it keeps bearer authentication active.

[plepola]

Could it be a Ruuvi Gateway automatic update that resets the settings?

However, I haven't had any problems other than the fact that, at certain intervals, there is a new Gateway integration that needs to be disabled.

[TheSomeMan]

I think HA integration is OK. Problem is RuuviGateway itself.

Hi @mrl0leak, I'm the developer of Ruuvi Gateway. I can assure you that the Gateway itself does not change the configuration and does not write changes to flash memory, it only happens when the Gateway receives an HTTP POST request. The only exception is if the configuration was corrupted in the flash memory (or erased by a long press on the Configure button), then the default configuration will be written when the Gateway is rebooted. Do you have any differences from the default configuration other than the use of bearer authentication that would allow you to determine if a full configuration reset was done?

[TheSomeMan]

Hi @akx , could you please look at the following log (it seems that some exceptions are not handled properly):

2023-02-21 10:18:51.091 ERROR (MainThread) [ruuvi_gateway] Unexpected error fetching Ruuvi Gateway 9C:2C data: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 111, in connect_tcp
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_sockets.py", line 213, in connect_tcp
    async with create_task_group() as tg:
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_sockets.py", line 164, in try_connect
    stream = await asynclib.connect_tcp(remote_host, remote_port, local_address)
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1691, in connect_tcp
    await get_running_loop().create_connection(
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1049, in create_connection
    sock = await self._connect_sock(
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 960, in _connect_sock
    await self.sock_connect(sock, address)
  File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 500, in sock_connect
    return await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 110, in connect_tcp
    with anyio.fail_after(timeout):
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
    raise TimeoutError
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 86, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 63, in handle_async_request
    stream = await self._connect(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 111, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/auto.py", line 29, in connect_tcp
    return await self._backend.connect_tcp(
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc)
httpcore.ConnectTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aioruuvigateway/api.py", line 16, in get_gateway_history_data
    resp = await client.get(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1757, in get
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 239, in _async_refresh
    self.data = await self._async_update_data()
  File "/usr/src/homeassistant/homeassistant/components/ruuvi_gateway/coordinator.py", line 37, in _async_update_data
    data = await get_gateway_history_data(
  File "/usr/local/lib/python3.10/site-packages/aioruuvigateway/api.py", line 31, in get_gateway_history_data
    raise CannotConnect from err
aioruuvigateway.excs.CannotConnect

[mrl0leak]

I think HA integration is OK. Problem is RuuviGateway itself.

Hi @mrl0leak, I'm the developer of Ruuvi Gateway. I can assure you that the Gateway itself does not change the configuration and does not write changes to flash memory, it only happens when the Gateway receives an HTTP POST request. The only exception is if the configuration was corrupted in the flash memory (or erased by a long press on the Configure button), then the default configuration will be written when the Gateway is rebooted. Do you have any differences from the default configuration other than the use of bearer authentication that would allow you to determine if a full configuration reset was done?

I'm not sure I understand correctly what you are asking...But same thing happened again yesterday Gateway setting for bearer authentication is cleared.

Not all my settings are default, only bearer authentication gets cleared. So I think there is no full reset done. Just can't keep that bearer authentication checkbox checked.

But my settings are Connection type: Wi-Fi (I think Ethernet is default, so it was not reset) Current firmware version v1.12.4 Auto update is set to Auto update (recommended setting, accept releases after they're 2 weeks old) Enable bearer authentication (Cleared again, pasted my original token, and HA starts working) Use Ruuvi Cloud or/and a custom server and configure more settings HTTPS: https://network.ruuvi.com/record MQTT(S): MQTT over unencrypted TCP + Server, Port, User and password for Mosuito Broker from my HA server (This is not default setting, but stays as it is) Allow sending statistics: https://network.ruuvi.com/status Use default set of NTP servers for time synchronisation Listen to Ruuvi sensors

[mrl0leak]

If I go and check configuration right after re-activating bearer authentication. It is on. It just gets cleared by itself after few days.

[akx]

@TheSomeMan

Hi @akx , could you please look at the following log (it seems that some exceptions are not handled properly):

I don't think there's an issue here – the salient point of the traceback is that fetching either timed out (i.e. the gateway didn't respond in the default timeout (5 seconds) or was cancelled.

[mrl0leak]

I'm using RuuviGateway with my WiFi, so I'm not able to select static ip settings for it. I'm also not able to control my too-secure-router DHCP server. This leads RuuviGateway to change IP address from time to time. Today I made some changes to furniture locations and swiched router off for some time. Gateway got new address and thats OK. But it would be very nice to be able to change IP for RuuviGateway integration in HA.

So I deleted existing integration in HA and installed it again, to setup it with new IP address. And for pleasant surprise all my RuuviTAGs revived. There was no need to configure TAGs again.

So now I'm waiting RuuviGateway configuring itself again. Because thats what it keeps doing every few days. Bearer authentication just does not stay on. It is quite easy to fix, but it is annoying to lose measurement history in HA every couple of days.

[akx]

And for pleasant surprise all my RuuviTAGs revived. There was no need to configure TAGs again.

This happy surprise is because the Ruuvi Gateway acts as a (somewhat limited and strange HTTP-based :grin:) Bluetooth adapter as far as Home Assistant is concerned. You'd get the same effect if you set up e.g. a Bluetooth adapter or ESPHome...

That aside, I'm sorry you're having this trouble - I really don't know what's happening, especially if it seems it's the Gateway firmware playing tricks...

[mrl0leak]

It has now been OK for almost a week, fingers crossed.

I have also tried RuuviTAGs with ESPHome and those seem to be very reliable, only problem with these ESP32-modules is that they suck with bluetooth reception. I'm not able to receive RuuviTAG signal through single sauna wall + 2 meters air space with ESP32-module. This is probably because of that aluminium foil layer in sauna wall insulation. RuuviGateway is awesome with finding those TAG signals. Gateway easily picks up my sauna TAG through multiple walls in between and 20m away.

RuuviGateway integration is "BlackBox" to me compared to ESPHome. With ESPHome I feel that I have full control over those ESP32 modules. RuuviGateway integration does not show me current configuration. And I'm not able to correct even IP-address from it after my annoying router has decided to change my RuuviGateways addess.

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.