Some devices cannot be controlled in Assistant SDK due to losing context of previous command

[MaRolM]

The problem

I recently set up the new Assistant SDK integration. It works beautifully except for one thing. As a disclaimer, this is incredibly cursed and is probably the fault of an external company setting things up extremely wrong.

I have a Conga robot vacuum. It uses a "chatbot" way of interacting via Assistant that I suspect is this soon to be deprecated api and well described by this forum post.

Finding this issue was hard in the first place as trying most commands failed, enabling debug log for the integration I saw this:

2023-01-23 00:00:19.524 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: ask Robot Conga to start cleaning
response: To let me read out that information, turn on personal results in the Google Home app.
I didn't get that. Can you say it again?

Things weren't working until I tried doing it in Spanish instead, despite the company advertising English support and commands. Oh well. After switching the language of the integration it then seemed to work and awaited more commands, but the next command sent via HA seems to be not recognized as part of the same "conversation"

2023-01-23 00:04:33.668 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: Quiero hablar con el robot conga
response: Bienvenido a usar Conga Home. Aquí hay algunas cosas que puede decir: 'inicie la limpieza', 'detenga la limpieza' o 'vaya a cargarse'.
2023-01-23 00:05:05.349 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: inicie la limpieza
response: Parece que ese dispositivo no está configurado aún. Abre los ajustes del Asistente para añadirlo.

Commands work on phone but again, they use this weird chatbot ui.

What version of Home Assistant Core has the issue?

2023.1.7

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Google Assistant SDK

Link to integration documentation on our website

https://www.home-assistant.io/integrations/google_assistant_sdk/

Additional information

Don't buy Cecotec

[home-assistant[bot]]

Hey there @tronikos, mind taking a look at this issue as it has been labeled with an integration (google_assistant_sdk) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `google_assistant_sdk` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign google_assistant_sdk` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

google_assistant_sdk documentation google_assistant_sdk source _{^{(message by IssueLinks)}}

[tronikos]

https://github.com/home-assistant/core/pull/85423 which is included in 2023.2 release allows multiple commands in the same context. See example in the documentation https://www.home-assistant.io/integrations/google_assistant_sdk

service: google_assistant_sdk.send_text_command
data:
  command:
    - "open the garage door"
    - "1234"

[tronikos]

@home-assistant close

[p4block]

@tronikos I've been trying to make it work with no luck so far. Using the new multiple commands feature via the Blueprint causes this

2023-02-26 00:39:22.593 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: Quiero hablar con el robot conga
response: Bienvenido a usar Conga Home. Aquí hay algunas cosas que puede decir: 'inicie la limpieza', 'detenga la limpieza' o 'vaya a cargarse'.
2023-02-26 00:39:27.167 DEBUG (MainThread) [homeassistant.components.google_assistant_sdk.helpers] command: Inicie la limpieza
response: ¿Disculpa?

It tells you what to say and yet it doesn't understand the message somehow.

We can get more insight via the new Google Assistant inside Home Assistant Conversations feature.

After messing with with it for a while it seems like the first command successfully enters into the Conga integration , but once "inside" it doesn't recognize any command it should be recognizing. Sending "exit" will exit from the Conga chatbot and become a normal Assistant-In-Conversations.

[tronikos]

Does the following command work? "ask Robot Conga to start cleaning"