Does piper support AMD GPU acceleration with rocm?

[eliranwong]

Does piper support AMD GPU acceleration with rocm?

[eliranwong]

I can see onnx runtime support AMD rocm, please read:

https://rocm.docs.amd.com/projects/radeon/en/latest/docs/install/install-onnx.html

But how to get it integrated into piper?

[mush42]

@eliranwong

From what I know, you need to modify and rebuild Piper, setting the execution-providers to your chosen ones. Also, you need to provide onnxruntime build with your providers.

Best Musharraf

[eliranwong]

Appreciate your reply and help. May I ask for more information about:

1. how to set execution-providers for rebuilding piper
2. how to provide onnxruntime build with your providers

[eliranwong]

Do you mean I need to manually edit this line:

https://github.com/rhasspy/piper/blob/078bf8a17e24ebb18332710354c0797872dcef6a/src/python_run/piper/voice.py#L53

[mush42]

@eliranwong

1. Since Piper doesn't provide a command-line option to set onnxruntime EP to ROCM, you need to modify Piper's C++ code to set it manually in the source.
2. Find a library onnxruntime.so built with ROCM EP, if pre-built binaries are not provided by Microsoft, you need to build it yourself.

[eliranwong]

So far, below is the easiest way that I found:

1. Install ONNX Runtime with ROCm Execution Provider reference: https://huggingface.co/docs/optimum/onnxruntime/usage_guides/amdgpu#22-onnx-runtime-with-rocm-execution-provider

# pre-requisites
pip install -U pip
pip install cmake onnx
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install ONNXRuntime from source
git clone --recursive  https://github.com/ROCmSoftwarePlatform/onnxruntime.git
cd onnxruntime
git checkout rocm6.0_internal_testing

./build.sh --config Release --build_wheel --update --build --parallel --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) --use_rocm --rocm_home=/opt/rocm
pip install build/Linux/Release/dist/*

2. Manually edit this line:

https://github.com/rhasspy/piper/blob/078bf8a17e24ebb18332710354c0797872dcef6a/src/python_run/piper/voice.py#L53

to:

providers=["ROCMExecutionProvider"]

I am open to better solution.

I would appreciate if the author of piper and support it directly, so that I don't need to manually edit the line.

Many thanks.

[eliranwong]

I read piper currently support --cuda argument. I would suggest to add --rocm argument to make piper better.