Petals Error: GPU is not available

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

https://petals.dev

MIT License

9.26k stars 526 forks source link

Petals Error: GPU is not available #398

Open SephReed opened 1 year ago

SephReed commented 1 year ago

I'm running a 660Ti, so I'm pretty used to it not playing well with other things. If there's something I can do, that would be great. Otherwise, this is just a jumping off point for anyone who gets the same error.

AssertionError: GPU is not available. If you want to run a CPU-only server.....

borzunov commented 1 year ago

Hi @SephReed,

Can you share your OS and if you're using Anaconda or Docker? If it's Anaconda, can you please try Docker instead?

I believe it's a CUDA/PyTorch installation issue, it will work once you make your PyTorch see GPU.

SephReed commented 1 year ago

I'm on Fedora and using Anaconda.

And I apologize for being obtuse, but I don't want to have docker on my computer.

borzunov commented 1 year ago

@SephReed, can you please share what this command prints?

python -c "import torch; print(torch.cuda.is_available())"