notify.google_assistant_sdk not working with IPv6

[TyzzyT]

The problem

According to the docs, a notify.google_assistant_sdk command should look like this:

service: notify.google_assistant_sdk
data:
  command: time for dinner

But this doesn't work, because the notify.something command requires the message parameter. So when you try to execute the code above, it returns the error message:

Failed to call service notify.google_assistant_sdk. extra keys not allowed @ data['command']. Got None required key not provided @ data['message']. Got None

What version of Home Assistant Core has the issue?

core-2023.1.0b0

What was the last working version of Home Assistant Core?

never

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Google Assistant SDK

Link to integration documentation on our website

https://rc.home-assistant.io/integrations/google_assistant_sdk#service-notifygoogle_assistant_sdk

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @tronikos, mind taking a look at this issue as it has been labeled with an integration (google_assistant_sdk) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `google_assistant_sdk` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign google_assistant_sdk` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

google_assistant_sdk documentation google_assistant_sdk source _{^{(message by IssueLinks)}}

[firstof9]

Also note that issuing a service call:

service: notify.google_assistant_sdk
data:
  message: time for dinner

Results in nothing occurring either.

[dshokouhi]

In this case it looks like the docs table suggests that message needs to be used but the example below the table shows command.

Also looks like even if message is used the service call does not work. https://discord.com/channels/330944238910963714/427516175237382144/1057752230457004053

[firstof9]

Just for completeness the follow didn't work either:

service: google_assistant_sdk.send_text_command
data:
  command: broadcast This is a test

[tronikos]

I'm fixing the documentation in https://github.com/home-assistant/home-assistant.io/pull/25434

Does the following work?

service: google_assistant_sdk.send_text_command
data:
  command: turn on all the lights

If it doesn't, are there any errors in the logs? I suggest following the documentation again and set up a new google cloud project.

If it does, do broadcast commands work when you send them directly to google assistant either from your phone or speaker using "OK Google broadcast this is a test"? Are your speakers in do not disturb mode? If they are, broadcast doesn't work. Is your home assistant server IP address in the same network as your speakers? I think they need to be in the same network for broadcast commands to work.

[TyzzyT]

Hey @tronikos, When I do a command like: turn on bathroom light, it performs the action and it works. But the notify.google_assistant_sdk doesn't work, when I send something like someone is at the frontdoor or dinner is ready, nothing is happening with my Google Home speakers. They are not in Do not disturb mode. There also is no error message shown in the log. HA and Google Speakers are on the same network.

[legolas0802]

Logger: homeassistant.components.script.prova_message Source: helpers/script.py:409 Integration: Script (documentation, issues) First occurred: 14:21:35 (1 occurrences) Last logged: 14:21:35

prova_message: Error executing script. Invalid data for call_service at pos 1: extra keys not allowed @ data['command']

[tronikos]

@TyzzyT do broadcast commands work when you send them directly to Google Assistant either from your phone or speaker using voice "OK Google broadcast this is a test"?

@legolas0802 you need to use message for the notify service and not command. E.g.

service: notify.google_assistant_sdk
data:
  message: time for dinner

[TyzzyT]

@tronikos yes, when I ask Google Assistant: Ok Google broadcast this is a test, it will record my voice and broadcasts it on the Nest Hub and Nest Mini's around the house.

[NickM-27]

Using the notify service with the message parameter it’s working as expected for me, may be an issue with the GCP setup?

[Mariusthvdb]

if I do

service: notify.google_assistant_sdk
data:
  message: dit is een test

it works for all of my Google devices but the Google hub

click button

huge lag

green button

silly sound "incoming broadcast, it says:"

dit is een test

[TyzzyT]

Using the notify service with the message parameter it’s working as expected for me, may be an issue with the GCP setup?

Other Google Home commands like: turn on bathroom light are working. So that should confirm that GCP setup is correct?

[legolas0802]

service: notify.google_assistant_sdk data: message: time for dinner

it doesn't work, while with the lights and more it works, even if I have a delay of about 15 seconds in executing the command

[tronikos]

If commands work but notify doesn't then it shouldn't be an issue with the GCP setup. What language are you using for the integration? I've only really tested it with en-US. Can you change it to that in the configure page and test? If you check the code in notify.py I'm just converting the notify message to a command, for English that's: broadcast 'message'. For different languages I'm replacing broadcast with the respective verb. Maybe that's wrong? It would be good to experiment with different commands calling the send text command service.

@Mariusthvdb it's strange that it works for all your speakers except your Nest hub. I have every speaker and display, including both generations of Nest hub and Nest hub max and all are working fine. Does the Nest hub work for you when you say OK Google broadcast 'message' to your phone or another speaker?

[TyzzyT]

I tried it with both en-GB and en-US, but notify/broadcast still doesn't work. Very strange... I'll test some more today and let you know!

[TyzzyT]

I did multiple tests with en-US as language setting:

1. Type in Google Assistant app on my phone: broadcast this is a test - works as text to speech
2. Speak to Google Assistant: broadcast this is a test - works as recording/playback audio
3. Use google_assistant_sdk.send_text_command: turn on Ceiling light bathroom - works and turns on the light
4. Use google_assistant_sdk.send_text_command: broadcast this is a test - green check in HA but nothing is happening
5. Use google_assistant_sdk.google_assistant_sdk: this is a test - green check in HA but nothing is happening

It might be worth noting that I've signed up for the public Google Home beta and currently using that, not sure if it's related.

[legolas0802]

Se i comandi funzionano ma la notifica no, non dovrebbe essere un problema con la configurazione di GCP. Che lingua stai usando per l'integrazione? L'ho testato davvero solo con en-US. Puoi cambiarlo in quello nella pagina di configurazione e test? Se controlli il codice in notify.py sto solo convertendo il messaggio di notifica in un comando, per l'inglese è: broadcast 'message'. Per lingue diverse sostituisco broadcast con il rispettivo verbo. Forse è sbagliato? Sarebbe utile sperimentare diversi comandi chiamando il servizio di comando di invio del testo.

@Mariusthvdbè strano che funzioni per tutti i tuoi altoparlanti tranne il tuo hub Nest. Ho tutti gli altoparlanti e i display, incluse entrambe le generazioni di Nest hub e Nest hub max e funzionano tutti correttamente. L'hub Nest funziona per te quando dici OK Google trasmette "messaggio" al tuo telefono o a un altro altoparlante?

use it-it is missing with en-us does not speak

[TAW-V]

I did multiple tests with en-US as language setting:

1. Type in Google Assistant app on my phone: broadcast this is a test - works as text to speech
2. Speak to Google Assistant: broadcast this is a test - works as recording/playback audio
3. Use google_assistant_sdk.send_text_command: turn on Ceiling light bathroom - works and turns on the light
4. Use google_assistant_sdk.send_text_command: broadcast this is a test - green check in HA but nothing is happening
5. Use google_assistant_sdk.google_assistant_sdk: this is a test - green check in HA but nothing is happening

It might be worth noting that I've signed up for the public Google Home beta and currently using that, not sure if it's related.

This is exactly the same for me, except i'm not on the public Google Home Beta.

Furhtermore, I have a 5-10 sec delay between submitting and green check / turning on the lights

[tronikos]

The large latency is expected. It's on Google servers, nothing I can do about it.

Can you follow the instructions in https://pypi.org/project/google-assistant-grpc/ to generate credentials.json using google-oauthlib-tool, checkout https://github.com/tronikos/gassist_text and run:

python3 -m venv .venv
source .venv/bin/activate
# for Windows CMD:
# .venv\Scripts\activate.bat
# for Windows PowerShell:
# .venv\Scripts\Activate.ps1

# Install dependencies
python -m pip install --upgrade pip
python -m pip install .

# Run command line interactive tool
python -m pip install click
python demo.py --lang=en-US

In the command line interactive tool type:

: broadcast this is a test

to see what the assistant's response is?

[tronikos]

Instead of following the previous instructions, if you update to 2023.1.0b4 and enable debug logging for the integration, you can inspect the debug log for the command and response.

[TyzzyT]

I did multiple tests with debug logging enabled, but I don't think it is of any help. The response is always: OK. Broadcasting now.

2023-01-02 08:21:29.608 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: broadcast testing out the broadcast message with debug logging enabled response: OK. Broadcasting now. 2023-01-02 08:21:36.347 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: broadcast this is a test response: Alright. Broadcasting now. 2023-01-02 08:21:51.000 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: broadcast testing out this new fancy Google Assistant SDK integration response: OK. Broadcasting now.

[piitaya]

I just did some tests :

1. From Home Assistant

It doesn't work.

Service call

service: google_assistant_sdk.send_text_command
data:
  command: "Annonce dans la cuisine: La fenêtre est ouverte" # Broadcast in the kitchen: The window is open

Logs

2023-01-02 10:18:03.350 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: Annonce dans la cuisine: La fenêtre est ouverte
response: OK. Je transmets ça à "Enceinte Cuisine". 

Translation of the response : "OK. I will broadcast this to "Kitchen Speaker".

2. From my phone using Google Home App

It works with the same text.

Speaker broadcast

Annonce de Paul : La fenêtre est ouverte. (Translation : Paul's broastcast: the window is open).

Maybe the name of the sender is missing when using HA service?

[mdehaas]

homeassistant.components.google_assistant_sdk

Is the following in configuration.yaml enough to enable the right debugging: homeassistant.components.google_assistant_sdk: debug

[piitaya]

You can activate debug from UI :

[TAW-V]

I updated home assistant to version 2023.1.0b4 and this is the debug log when I try to use notify.google_assistant_sdk:

2023-01-02 16:17:20.889 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: broadcast The garage door has been open for 10 minutes. response: Alright. Broadcasting now.

Nothing plays on my google speakers. On my phone however, broadcasting a message to my speakers does work.

When I try to turn some lights off via google_assistant_sdk.send_text_command, the correct lights turn off. However, debug message says that google cannot reach home assistant:

2023-01-02 16:22:08.038 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: turn off the TV lights response: Sorry, I couldn't reach the Home assistant.

[RyanEwen]

Here to say I'm having the same issue.

Can issue commands without a problem. Can broadcast using Google Assistant on phone and Google Homes. Cannot broadcast from HA.

[isewise]

I have tested commands and they work. Slow, but they work.

However I have not been able to broadcast using the notify.google_assistant_sdk service call.

I turned on debug for Google Assistant SDK and it returns this after I submit a notify.google_assistant_sdk service call:

2023-01-04 15:09:33.081 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: broadcast to bedroom Test response: Alright. Broadcasting to "bedroom".

But I hear nothing from my Google Home mini. On same wifi, its not in Do Not Disturb mode,

I have also tried spelling bedroom differently with an uppercase Bedroom. Still nothing.

Checked the GCP dashboard and it states all 200s for the API.

I am in the new Google Home Beta.

[tronikos]

Broadcast is working fine for me even though I'm in the Google Home Beta. I'm out of ideas why it's not working for you. Can you try my earlier suggestion in https://github.com/home-assistant/core/issues/84726#issuecomment-1368118599 but with: python demo.py --lang=en-US --display to see if the HTML response has any useful information?

[Cameronwyatt]

Adding another datapoint: I upgraded to 2023.1.0 today and was able to get broadcasting using notify.google_assistant_sdk working just fine. However, every message that's sent is prefaced with "Speaker".

[donparlor]

Same problem here, my Google Home and Home Assistant both runs in french-canada. When I send a message with notify.google_assistant_sdk, it does connect with the target specified, but it won't broadcast the message....

[heroxy2111]

Same problem with de-de setting. Same network, no do-not-disturb. Can broadcast from my phone but not from HA.

[andrei-micu-ro]

I think this is a general behavior that everyone has. It's the same for me - all commands work except for the notification ones. From assistant directly (from phone or tablet) it works Google Console shows all requests without errors (even the notification/broadcast ones). I wish I could sniff the actual requests that google receives to find any differences...

The sad part is that the command properly reaches Google servers as I can see in "Activity" that it responds "Ok, broadcasting now" but nothing happens.

I fear Google is ignoring the request because it has "something" different than what it's expecting and eats the error in the process. Is there a Python script I can run manually from the HA host machine to test this ?

[dziadeck47]

For me the notification works if the message is spelled properly in the language set up in Google Assistant SDK (i live in Poland, Polish is supported by assistant but not by ghome or by SDK, my language is set to en-us I believe. 'Message: new message' works but e.g. 'message: nowa message' does not. Also the SDK commands are extremely limited to turn on/off, volume 2 in Nest etc. Stream sth on Nest etc does not work. And I had such hopes for it :(

[gregg098]

This all seems similar to the issues we used to have with the old addons. See THIS thread for reference (Note: ITS LONG). I honestly dont know what's changed since that addon was created vs this SDK addon, but seems as if the issues are the same.

Basically there were two main issues:

1) If you had IPV6 enabled, broadcasts did not work. I ran into this years ago. I dont have the luxury of turning IPV6 off to test, but might be worth someone who can experiment. 2) Regardless of IPV6, If your Google devices and your Home Assistance instance were not on the same network (or VLAN), broadcasts did not work.

Again, I'm lacking any additional knowledge here, but seems like the same issue fought years ago with no fix.

[donparlor]

Basically there were two main issues:

1. If you had IPV6 enabled, broadcasts did not work. I ran into this years ago. I dont have the luxury of turning IPV6 off to test, but might be worth someone who can experiment.
2. Regardless of IPV6, If your Google devices and your Home Assistance instance were not on the same network (or VLAN), broadcasts did not work.

Again, I'm lacking any additional knowledge here, but seems like the same issue fought years ago with no fix.

My IPv6 is OFF on my Google Nest Wifi Mesh Network, and my HA and Google Devices are on the same network. It still does not broadcast.

[gregg098]

Basically there were two main issues:

1. If you had IPV6 enabled, broadcasts did not work. I ran into this years ago. I dont have the luxury of turning IPV6 off to test, but might be worth someone who can experiment.
2. Regardless of IPV6, If your Google devices and your Home Assistance instance were not on the same network (or VLAN), broadcasts did not work.

Again, I'm lacking any additional knowledge here, but seems like the same issue fought years ago with no fix.

My IPv6 is OFF on my Google Nest Wifi Mesh Network, and my HA and Google Devices are on the same network. It still does not broadcast.

If I remember correctly, with the old Google Relay addons, I could only ever get it to work with IPV6 100% disabled at the router level. I experimented with non-IPV6 VLANs and all kinds of weird configurations. Never worked. But again, someone who knows better please correct me if I'm missing anything here or if this is not even relevant to the current implementation.

[isewise]

I just turned off IPV6 at the router and it started to work immediately.

Sounds like YMMV

[RyanEwen]

It works for me after turning off IPv6 and restarting my network (Nest Wifi Pro). Not ideal, but at least it works

[blair287]

On 2023.01 text commands working broadcast via text command or the notify isn't working nothing happens no errors etc.

[tronikos]

Thanks for bringing up IPv6. I'm updating the documentation in https://github.com/home-assistant/home-assistant.io/pull/25581 to recommend users disabling it. The integration is using the exact same Google Assistant Service that the assistant relay and the various addons are using so they all have the same issues.

[maennes]

At first, I could use the service _google_assistant_sdk.send_textcommand but was unsuccessful with service _notify.google_assistantsdk. Debug messages indicated that the device (Nest Hub) could not be found.

I then made the following changes that seem to have fixed this issue. I don't know if both actions were needed.

1. In HA, I removed the device from the cloud sync at /config/cloud/google-assistant
2. I factory-reset the device

[Blogshot]

I managed to at least get commands working by checking this thread and re-enabling my test app on the top right of the page.

If you've already setup some Google API it is possible that you have to re-enable the "My test app". Now commands work, only broadcasts are not played despite the debug log acknowledging them without issue.

I'm not willing to disable IPv6, so I'll stick with the commands for now.

[silvannn]

Disabling IPv6 on router fixed all my problems with broadcasting. Is it possible to turn off Google message "incoming broadcast. it says:". I was exited too early.

[andyxpert]

Had the same problem with broadcasts not working, disabled IPv6 from the router and now it works like a charm.

It is strange as the google devices could be reached by Google servers from Google apps even with IPv6 turned on, so it's not a networking issue. I suspect some extra security on referral URIs synced with target URIs on Google servers when Assistant API is used...

Anyway, disabling IPv6 fixed it

[isewise]

Is it possible to turn off Google message "incoming broadcast. it says:". I was exited too early.

No, it's part of the broadcast feature.

[silvannn]

Is it possible to turn off Google message "incoming broadcast. it says:". I was exited too early.

No, it's part of the broadcast feature.

Ohh. I was thinking to switch from google tts to this feature to have music resumed after announcement and to have more sexy female voice instead of robotic tss but I do not like this silly message.

[marcelveldt]

The issue with notifications not working if IPv6 is enabled is a known issue. Each and every project based around the Assistant SDK has this same issue. Disabling IPv6 is imo a very bad idea in 2023. The Assistant SDK is a pretty old project not seen a lot of updates lately and I doubt if this particular issue is going to be fixed. Just to set some expectations here.

[andrei-micu-ro]

True, such networking constraints shouldn't happen in 2023, but seems like a reasonable temporary workaround for people using this specific scenario. I did worse workarounds and adaptations to make some Meross-based sensors work with HA, hardcoded IDs in python drivers, etc. It would be nice though if Google would open the assistant backend more and design better APIs so that the Open Source community can better integrate with it.

[marcelveldt]

Well, sure but I have to warn you that is already the workaround for the last 5 years or so...

[flecmart]

There has to be another solution than disabling ipv6 completely. Damn I was really happy seeing the google assitant sdk officialy added to homeassistant, but disabling ipv6 is just not a really sustainable solution...