Wyoming whisper support for Jetson AGX Xavier

[ark626]

If nobody has until now i wanted to push support for older models like AGX Xavier for the Docker containers like wyoming whisper.

Especially Wyoming whisper might profit from getting a native docker image on Jetson AGX Xavier because currently we can only use CPU when using whisper. Having a container like https://hub.docker.com/r/dustynv/wyoming-whisper for other platforms would be appreciated. Altough i haven't managed to get it running it seems there are already many Docker Images for GPU use and Wyoming Whisper. See https://github.com/andyattebery/faster-whisper-cuda-docker

[dusty-nv]

Hi @ark626, are you currently able to build it for JetPack 5? IIRC at the time those containers were added, I encountered compilation errors in the HA/Wyoming stack on JP5. But if they build, I will push the images to dockerhub :+1:

[ark626]

Hi @dusty-nv

i tried to get it running but it seems the CTranslate2 package is needed with Cuda. So i tried compiling but this doesn't seem to work for me.

The rest seems running altough i cannot confirm i CTranslate2 is the only missing thing here. If we could compile the CTranslate2 properly for the Jetson devices it should run but im not really familiar with this.

I have added a forked a Dockerconfig and tried to adapt it so it works: https://github.com/ark626/faster-whisper-cuda-docker.git Don't use the docker-compile file but instead do:

cd cuda nvidia-docker build . -f Dockerfile --build-arg COQUI_BRANCH=v0.16.5 -t jetson-coqu docker run --runtime nvidia -it --rm --network=host jetson-coqu -p 10300:10300 -p 10300:10300/udp --name coqu

Maybe this was the reason why it didn't work in the first place.