search

152334H / DL-Art-School

TorToiSe fine-tuning with DLAS
GNU Affero General Public License v3.0
205 stars 86 forks source link

Error no kernel image is available for execution on the device #75

Open consensual-hallucination opened 1 year ago

consensual-hallucination commented 1 year ago

Hi, when I try to train, it throws an error: 

C:\Users\PC\anaconda3\envs\DLAS\lib\site-packages\torch\optim\lr_scheduler.py:138: 
UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. 
In PyTorch 1.1.0 and later, you should call them in the opposite order: 
`optimizer.step()` before `lr_scheduler.step()`.  
Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. 
See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
Error no kernel image is available for execution on the device at line 167 in file 
D:\ai\tool\bitsandbytes\csrc\ops.cu
Press any key to continue

I thought this may be wrong CUDA version installed but I checked and my 1070 seem to be compatible with with CUDA 8+.

Anyone encountered this error and found a way to fix it? Thanks.

mementos94 commented 1 year ago

I have the same issue with 1070ti

consensual-hallucination commented 1 year ago

I have the same issue with 1070ti

The issue seem to be bitsandbytes / cuda version related. I didn't have much time to investigate it yet so for now I'm using DLAS without it and it works ok. It would be nice to have it working tho. I'll look into this once I have some spare time.