search

GrapheneOS / os-issue-tracker

Issue tracker for GrapheneOS Android Open Source Project hardening work. Standalone projects like Auditor, AttestationServer and hardened_malloc have their own dedicated trackers.
https://grapheneos.org/
361 stars 21 forks source link

Support offline speech-to-text via sandbox service #1593

Closed lake-effect closed 2 years ago

lake-effect commented 2 years ago

This is somewhat ambitious, but the enhancement here is to support offline speech-to-text via keyboards like Gboard.

The setup is spread out over multiple apps:

  1. Install the Google app (not available in GrapheneOS, but can be sideloaded)
  2. Install Gboard, configure permissions as usual for keyboard
  3. Enable voice typing in Gboard
  4. Once the Google app is installed, offline dictionary download is available in Gboard's voice typing settings
  5. Enable Gboard network permission
  6. Download voice recognition models for languages of your choice
  7. Enable microphone permission for Gboard, disable network permission
  8. Press microphone button on Gboard and speak into phone (Bluetooth microphones not supported?)

The catch is that uninstalling the Google app removes all of the offline model settings again, and requiring network access to convert speech to text. It would be much better to isolate whatever the Google app is providing to Gboard, and ensure that is always available.

There is no other set of models that supports as many languages or does so for free, so it's a strong feature.

thestinger commented 2 years ago

I don't understand what this is requesting. We aren't going to modify specific apps.

lake-effect commented 2 years ago

What I am proposing is bringing whatever the Google app provides into the GSF sandbox or similar so that users who need speech-to-text have that as an accessiblity option.

thestinger commented 2 years ago

You can try using Google's speech services app. It will sandboxed like any other app you install. It may already work fine.