Sending Text Commands

[Nerwyn]

Is your feature request related to a problem? Please describe. The ability to send text rather than using the TV keyboard would make using this remote even better than the physical one. It should be possible to use the existing Home Assistant integrations to send keystrokes to Android TVs.

Describe the solution you'd like The ability to send keystrokes to Android TV.

Describe alternatives you've considered The Google TV app has the best implementation of this, as it seems to live capture the contents of the text box you're interacting with and modify it in real time. I'm not sure how they are doing that or if it's part of Android TV Remote Protocol V2.

Additional context Using the alphabetical keycodes listed here with remote.send_command frustratingly does nothing. I'm not sure if this is a bug in the integration, the underlying library, or Android TV Remote Protocol V2.

It is possible to send text to Android TVs using Home Assistant via ADB, but it isn't nearly as graceful as doing so from the Google TV app. I created an input_text entity and a script that when called, would send the contents of the textbox to the Android TV.

alias: Lounge TV - Google TV Send Text
sequence:
  - service: androidtv.adb_command
    data:
      command: input text "{{ states("input_text.lounge_tv_google_tv_text_input") }}"
    target:
      entity_id: media_player.lounge_google_tv_adb
mode: single
icon: mdi:form-textbox

And a lovelace config to go with it that includes backspace functionality:

type: entities
entities:
  - entity: input_text.lounge_tv_google_tv_text_input
    hold_action:
      action: call-service
      service: script.lounge_tv_google_tv_send_text
    tap_action:
      action: call-service
      service: remote.send_command
      data:
        entity_id: remote.lounge_google_tv
        command: KEYCODE_DEL

This method works but feels hacky and inelegant. I want to try to find a better solution, like one that would transmit keycodes to the Android TV live rather than in bulk.

[Nerwyn]

As of v1.4.0 this feature has been implemented using the ADB integration! But, it could be better. Currently the keyboard button opens a browser text prompt, in which text can be entered and sent to the TV. While this works I want to work on a more seamless solution in the future using a keypress event listener. I'll need a good way to trigger opening the keyboard from JS when a text box is not present, and to create translations for all JS keyboard event codes to ADB keyboard event codes. Or I could keep doing ADB input text commands and have special cases for non text keys like backspace/delete.

[dnestico]

Ok so after restarting my Google TV I got this to kinda work, so I am now able to send text commands to the search bar, but the enter button on the card unfortunately doesn't work for me. Maybe its a Google TV problem? Not sure what to do.

[Nerwyn]

When you select the text box, does the on screen keyboard show up? I wonder if this is related to a known issue with the virtual keyboard not showing up when using an API remote. After some more testing it looks like the enter button only searches when the on screen keyboard is visible.

[dnestico]

For me nope the on-screen keyboard does not show up, apparently, this virtual keyboard can be disabled in settings but then that would probably just stop this whole feature from working.

[Nerwyn]

Disabling the virtual keyboard may actually help according to this thread. I'll try to find a better solution in the future. Maybe there's a way to trigger search without the onscreen keyboard via the remote API or ADB.

[dnestico]

What about this upcoming feature in 2023.8, idk what it fully does but will this also help? https://github.com/home-assistant/core/pull/95765

And alright I will disable it later and see what happens.

[Nerwyn]

Hopefully. I forget how exactly I fixed the issue for me a while back. Can you share a picture or list of what your enabled keyboards looks like? It should be under Settings > System > Keyboard > Manage Keyboards. Here's what mine looks like. Only Gboard is enabled and all others are disabled.

[Nerwyn]

I disabled Gboard on one of my Google TVs and tried to trigger a search without the keyboard but wasn't able to do it, not even with the Google TV app's keyboard. I think it may be a limitation with Android TV that the only way to do a text search is the search button on the virtual keyboard.

[Nerwyn]

I did find this buried in the HA ADB documentation. I think I can add an alternate search button to do a global search using it with this ADB command:

am start -a "android.search.action.GLOBAL_SEARCH" --es query "{{ states('input_text.androidtv_search_input')}}"

[Nerwyn]

@dnestico while I can't fix the no keyboard showing up and therefore can't actually click enter/search issue, I did find a workaround and put it in v1.4.1. It should let you use global Google Assistant search instead. Let me know if it works!

[dnestico]

Ok I will update when it is available, sorry I went out to the theatres to watch a movie so couldn't answer.

1. Do I still need to disable the virtual keyboard with v1.4.1?
2. Is global Google Assistant search like voice search btw?

[Nerwyn]

1. Do I still need to disable the virtual keyboard with v1.4.1?

If you want to use regular text input within apps yes. For global/Google Assistant search no.

2. Is global Google Assistant search like voice search btw?

Yes, it's the same. It's called global search in the ADB call but it opens the Google Assistant search within Google TV.

[dnestico]

Updated and added the Google Assistant button, haven't tested yet but here is my virtual keyboard settings btw I turned off virtual keyboard like you said, GBoard can't be disabled for me so whatever but hopefully this fixes things next time I try it out 👍

[Nerwyn]

Seamless keyboard support has been added with release v1.5.1 (latest at writing is v1.5.3). Home Assistant has also added a work around for the on screen keyboard (and therefore the ability to trigger search) not showing up as described here:

Some onscreen keyboards enabled by TV manufacturers do not support concurrent virtual and onscreen keyboard use. This presents whenever a text field is selected, such as “search” where a constant use the keyboard on your mobile device will show, preventing you from opening the onscreen keyboard to type. This can be overcome by either disabling your 3rd party keyboard and using the default Gboard keyboard or by unselecting Enable IME in the Configure page of the integration.