search

CiaraStrawberry / svd-temporal-controlnet

410 stars 31 forks source link

"exp_vml_cpu" not implemented for 'Half' #7

Open weijiawu opened 8 months ago

weijiawu commented 8 months ago

Hello, author. Thank you for the great open-source project. I encountered the following bug while training the code. Do you have any suggestions?

File "train_svd.py", line 1425, in <module> main() File "train_svd.py", line 1172, in main encoder_hidden_states = encode_image( File "train_svd.py", line 1031, in encode_image pixel_values = _resize_with_antialiasing(pixel_values, (224, 224)) File "train_svd.py", line 253, in _resize_with_antialiasing input = _gaussian_blur2d(input, ks, sigmas) File "train_svd.py", line 333, in _gaussian_blur2d kernel_x = _gaussian(kx, sigma[:, 1].view(bs, 1)) File "train_svd.py", line 320, in _gaussian gauss = torch.exp(-x.pow(2.0) / (2 * sigma.pow(2.0))) RuntimeError: "exp_vml_cpu" not implemented for 'Half'

BlakeOne commented 8 months ago

For now you can try it without this argument and that should fix the error. But fixing this option FR could allow training with less GPU RAM so it's still worth sorting out at some point IMO...

--mixed_precision="fp16"

pokameng commented 8 months ago

@CiaraStrawberry @weijiawu @BlakeOne same problem RuntimeError: "exp_vml_cpu" not implemented for 'Half'

pokameng commented 8 months ago

@weijiawu hi ,can we chat with each other from wechat? i am reproducing for this repo

BlakeOne commented 8 months ago

@pokameng Did you try training without the argument --mixed_precision="fp16? That fixes the error for me...

pokameng commented 8 months ago

@pokameng Did you try training without the argument --mixed_precision="fp16? That fixes the error for me...

yes I am training without the argument

weijiawu commented 8 months ago

@weijiawu hi ,can we chat with each other from wechat? i am reproducing for this repo

sure, my wechat is 15167756092