Whisper-based Voice Keyboard

TODO! I wrote this before i was made aware of whisper.cpp. Need to integrate that, into this, or build a new one on that.

Integration of the OpenAI speech to text model into Android.

Preparation

Prerequisites

• Rust
  • Rust Android Targets
    • rustup target add aarch64-linux-android # for arm64
    • rustup target add x86_64-linux-android # for x86_64
  • Cargo NDK

Tools

• Android Studio
  • Tools -> SDK Manager -> SDK Tools ->
    • Android SDK Build-Tools 33.0.0
    • NDK 25.1.8937393
• VSCode
  • Extensions
    • Rust Analyzer

Assets (Included)

• whisper.tflite TFLite Model + filters_vocab_gen.bin Mel Filters and Vocab
  • How it was made
  • Where I found it
  • Where I got it
• libswscale.so libswresample.so libavutil.so libavformat.so libavfilter.so libavdevice.so libavcodec.so FFMPEG for android
  • How to build it

How to run

• Step 1: VSCode:
  • Open the vscode project at /WhisperVoiceKeyboard/app/src/main/rust and verify the environment variables set in .vscode/settings.json/rust-analyzer.server.extraEnv and .vscode/cargoNdkEnv.sh
  • run ./.vscode/buildAll.sh to build the rust artifacts.
• Step 2: Android Studio: Just open the project. Need to expand project view to see rust files. Primarily for editing the Android App source.