Closed ZFTurbo closed 9 months ago
My current dirty fix is:
@property
def device(self):
# return next(self.parameters()).device
return 'cuda'
@ZFTurbo hmm that's weird, not sure why it can't find any parameters
could you retry with distributed data parallel and see if this issue persists?
@ZFTurbo try 0.3.9?
New code works. Thank you!
yup no problem
highly recommend huggingface accelerate btw! it will take away all the pain of doing distributed training
There is a problem with model if you try to use multiGPU:
What is the correct way to fix it?