Local query not being triggered

[ggear]

Environment:

• app.js Version: 2.1.6
• HAAS Version: 2023.3.6
• Google Nest Mini Gen2/Hub Gen2 Version: 1.56.324896
• Local HTTP deployment reverse proxied through nginx over TLS (not NabuCasa)

As above, I have configured each of the components as per the instructions and when debugging the google side, I can see successful IDENTIFY and REACHABLE_DEVICES intents but the QUERY and EXECUTE handlers are never invoked, despite trying multiple permutations of the following:

• Google device isolation and reboots (I have tried all, one type and singular Mini’s/Hub’s)
• HAAS configurations (report_state true/false, all, few or singular exposed entities, multiple and out of order reboots and SYNCs)
• Javascript debugging (adding PROXY_SELECTED, additional logs, reverting to simple examples)

The Google devices seem to have flagged each reachable device as without a local path and always fall back to cloud, without even trying. This is despite the fact they happily connect to the HAAS instance via the MDNS announcements, which are fully resolvable from devices on the same network. My nginx reverse proxy sees the fallback EXECUTE and multiple QUERY requests from the Google cloud at each reachable device operation. No attempt is made to run these locally. If I disable Support local query in my google action, the fallbacks occur even when the Google device is placed into LOCAL mode. If Support local query and LOCAL mode are both enabled, the Google device reports that each reachable device is not available and I have to return the Google device to DEFAULT mode to have the fallback work again.

I note that others have had this issue with the Google firmware that I am on (1.56.324896) here and not when at least one device was on an older firmware, perhaps that is the issue? Can anyone confirm they have local fulfilment working with this (and only this) Google firmware?

So to me, it looks like there has been some behaviour change on the Google side and the question is if we can code around that or not:

• Is there some way to flag the reachable devices for local query/execute and not cloud? Perhaps some combination of the Google interface willReportState and willReportStateViaPoll options and or HAAS backend state handler push implementation can resolve this (we know the google IP devices after IDENTIFY, perhaps we bypass the Google cloud entirely?)
• Perhaps not implementing the PROXY_SELECTED intent is flagging the hub as not available for the proxied reachable devices so Google does not attempt to resolve a local path? I hacked out a ProxySelectedPayload.proxyData to no avail and after not being able to see where the Google Local SDK used it outside of caching for the client, I abandoned it, but might be worth revisiting.
• Something else I am missing? I have lots of reachable devices, perhaps I am exceeding a limit? Perhaps my MDNS environment is screwy?

Any thoughts are appreciated, I am highly motivated to resolve this issue, it is the last weak cloud link I have in my environment and I am keen to optimise it as much as possible.

[ggear]

Also this report seems to show I am not alone with this issue, since the start of 2023.

I should also note that although I had configured for local fulfilment for some time before this, but given it seemed to just work, I never looked into that much until I recently was investigating further. I presume it was previously working, but it could have just been falling back to cloud operation, so I can’t be sure.

Local fulfilment seems very finicky, which is a shame, seems like such an opportunity!

[Caprico85]

I can't really speak for NabuCasa. But mikejac/node-red-contrib-google-smarthome and andrei-tatar/node-red-contrib-smartnora are having the same problem.

This seems to be a problem on Google's side. A fix is expected mid-May. See https://issuetracker.google.com/issues/261893570#comment31

[ggear]

Thanks @Caprico85, that looks like the issue, I appreciate the heads up.

I may investigate how to block my Google devices from upgrading their firmware in the future once I get software stable, given Google’s poor responsiveness.

[Caprico85]

Google released a new firmware for their speakers, which fixed local execution. At least for node-red-contrib-google-smarthome. I guess it's the same for NabuCasa.