Coral TPU support

[ciaotesla]

I wanted to know if there is a way to add Google Coral TPU support. I wanted to experiment with Raspberry PI 5 and Coral TPU.

[joshuachris2001]

bump, as it would be interesting.

[TheTechmage]

I was interested in this as well and gave it a shot. Following the install instructions for Google Coral USB Accelerator, I setup a python virtual env for version 3.9.12. Once I verified that the Coral was working and it could identify birds, the next step was getting whisper-jax up and running. Doing so involves the following steps:

• Installing jax[tpu]
• Follow directions to setup whisper-jax as an endpoint
• modify app/app.py to use openai/whisper-base.en or openai/whisper-tiny.en. Anything larger uses a ton of swap space (I got up to 12 GB of swap in one of my tests)

Fortunately, even with the smaller models, it appears to work. Though it can take quite a while to do transcriptions with the single USB Accelerator TPU (2 and a half hours for an hours worth of audio)