[FEAT] Make cuda available

mricharz commented 2 months ago

Is this a new feature request?

[X] I have searched the existing issues

Wanted change

I would like to have the ability to activate cuda as it is supported by piper itself. The piper documentation says we need to install onnxruntime-gpu and add --cuda on the start script.

Reason for change

Piper should work much faster with cuda support on nvidia cards.

Proposed code change

Install onnxruntime-gpu in docker container.
Add a variable to docker image with will be passed to the start script like EXTRA_PARAMS With such a variable the user could add additional parameters to the piper start script as they like.

github-actions[bot] commented 2 months ago

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

LinuxServer-CI commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

4-FLOSS-Free-Libre-Open-Source-Software commented 1 month ago

    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    runtime: nvidia # https://docs.linuxserver.io/images/docker-plex/#nvidia

Append --cuda cmd there https://github.com/linuxserver/docker-piper/blob/main/root/etc/s6-overlay/s6-rc.d/svc-piper/run#L7-L16 ?

LinuxServer-CI commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

mricharz commented 1 week ago

    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    runtime: nvidia # https://docs.linuxserver.io/images/docker-plex/#nvidia
Append --cuda cmd there https://github.com/linuxserver/docker-piper/blob/main/root/etc/s6-overlay/s6-rc.d/svc-piper/run#L7-L16 ?

Yes. Should be the correct place.

mricharz commented 1 week ago

You can find the documentation here

If you'd like to use a GPU, install the onnxruntime-gpu package:
.venv/bin/pip3 install onnxruntime-gpu
and then run piper with the --cuda argument. You will need to have a functioning CUDA environment, such as what's available in NVIDIA's PyTorch containers.

srvoets commented 2 days ago

You can find the documentation here
If you'd like to use a GPU, install the onnxruntime-gpu package:
.venv/bin/pip3 install onnxruntime-gpu
and then run piper with the --cuda argument. You will need to have a functioning CUDA environment, such as what's available in NVIDIA's PyTorch containers.

yes you can do that in the console, but it needs to survive updates etc.

I found this due to the same issue.

linuxserver / docker-piper