search

johnsmith0031 / alpaca_lora_4bit

MIT License
534 stars 84 forks source link

curious on the finetuning speed of single GPU card #89

Open wesleysanjose opened 1 year ago

wesleysanjose commented 1 year ago

finally i am able to finetune using some dataset. but it looks like it will take 60 hours to complete 1 epoch on my 2060s (8GB vram). does it sound right?

image image
tensiondriven commented 1 year ago

If I'm reading that right, it looks like your GPU is using about 5Gb out of 8GB, which could be normal, but what's really weird is the temperature is 52, and it looks like the GPU utilization may be zero. I would expect the temp to be in the 70's and to see higher utilization.

Is it possible that you're running in CPU mode?

Also, how large is your training data? I think this has an effect on how long each epoch takes.

Another thing you can try is doing a training run in 8-bit using text-generation-webui and comparing the speed.

wesleysanjose commented 1 year ago
image

here you can see GPU is full throttle, why temp was low is because my PC case was removed :)

wesleysanjose commented 1 year ago

is it because monkeypatch slowing down the finetuning speed?

wesleysanjose commented 1 year ago

created a colab file and the finetune is even slower, the slowness doesn't come from my own setup

image