simonw / til

Today I Learned
https://til.simonwillison.net
Apache License 2.0
1.08k stars 85 forks source link

Dolly not using GPU #69

Open UtahDave opened 1 year ago

UtahDave commented 1 year ago

I ran into the issue you hit where the GPU isn't being used by pytorch.

First you need to find out which version on cuda is installed, then you need to install pytorch kind of like this:

sudo pip install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113

Change the torch version to what you want and change the cuda version to match what's installed in your environment