search

matatonic / openedai-speech

An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.
GNU Affero General Public License v3.0
473 stars 59 forks source link

Support Python 3.12, Torch 2.4, CUDA 12.4 #39

Open C0rn3j opened 3 months ago

C0rn3j commented 3 months ago

Torch 2.3.x only supports CUDA 12.1
Torch 2.4, which came out 3 weeks ago (current Docker builds are older), supports CUDA 12.4 and also Python 3.12 Upcoming Torch 2.4.1 seems to support CUDA 12.5.

I see the dependencies simply specify latest torch, so maybe just rebuilding the current image and pushing an update would be enough?

EDIT: Apparently I am ignorant for thinking you need to match host CUDA version to the torch version, newer CUDA on host seems to work fine with older CUDA on apps and was running into a Docker documentation issue with not being able to get it to run, but updating the image would still be nice.

matatonic commented 3 months ago

Is there any issue with older torch or CUDA here or just a request for updated versions? I update as far as I can normally, but some things have tough dependencies.

C0rn3j commented 3 months ago

Not really,
I realized you can have a newer CUDA on host and just run with older CUDA on the actual apps too late, otherwise I would have not made this issue.

2.4.0 does add Python 3.12 compatiblity though, which some people would probably welcome as modern distros now ship with that.

matatonic commented 3 months ago

I'll leave this open because it should be done eventually, but for now it's not going to happen, I've already tried upgrading to torch 2.3 and had a host of problems.