search

RustAudio / deepspeech-rs

Rust bindings for the deepspeech library
Other
295 stars 24 forks source link

How do I make deepspeech-rs use my GPU? #24

Closed nihiluis closed 4 years ago

nihiluis commented 4 years ago

Or does it already?

est31 commented 4 years ago

This is probably best asked in the upstream repo as this project only wraps the API they provide: https://github.com/mozilla/deepspeech

It seems it's still an open issue for them: https://github.com/mozilla/DeepSpeech/issues/2270

nihiluis commented 4 years ago

so this lib uses TF lite?

I thought you would use a specific gpu package as shown in the README.md upstream.

# Create and activate a virtualenv
virtualenv -p python3 $HOME/tmp/deepspeech-gpu-venv/
source $HOME/tmp/deepspeech-gpu-venv/bin/activate

# Install DeepSpeech CUDA enabled package
pip3 install deepspeech-gpu
est31 commented 4 years ago

I guess that part is for the training. This wrapper is about the inference part only.

nihiluis commented 4 years ago

So this library does not use Tensorflow Lite?

That was not the full snippet. This looks like inference to me. Check this: https://github.com/mozilla/DeepSpeech

# Create and activate a virtualenv
virtualenv -p python3 $HOME/tmp/deepspeech-gpu-venv/
source $HOME/tmp/deepspeech-gpu-venv/bin/activate

# Install DeepSpeech CUDA enabled package
pip3 install deepspeech-gpu

# Transcribe an audio file.
deepspeech --model deepspeech-0.6.1-models/output_graph.pbmm --lm deepspeech-0.6.1-models/lm.binary --trie deepspeech-0.6.1-models/trie --audio audio/2830-3980-0043.wav
est31 commented 4 years ago

Hmm yeah it might be possible to use this wrapper with a GPU. You could try it out with the cuda native client library in the release announcement. It's named like native_client.amd64.cuda.linux.tar.xz. I think the default builds use TF lite but it might be that these builds use proper tensorflow. IDK. Again, please ask upstream instead, they know more about how the native client is structured. This crate mainly wraps the high level parts of the API. Thanks.

lissyx commented 4 years ago

Hmm yeah it might be possible to use this wrapper with a GPU. You could try it out with the cuda native client library in the release announcement. It's named like native_client.amd64.cuda.linux.tar.xz. I think the default builds use TF lite but it might be that these builds use proper tensorflow. IDK. Again, please ask upstream instead, they know more about how the native client is structured. This crate mainly wraps the high level parts of the API. Thanks.

Yes, you can jsut use libdeepspeech.so built with TensorFlow + CUDA support as linked. Default builds are TFLite on platform that makes sense (Linux/ARM*, Android)