Rbrq03
commented
2 months ago Hello @chenjingcheng Thanks for interest!
I also find this bug during my experiment, so I think it's a bug in the original diffusers code. A quick fix is to unable the mix-precision in config of accelerate. You can use accelerate config to choose the config that fit to your local environment., check Accelerate for more details.
Also, I will add fixing the bug in code to my plan.
If you continue to experience this issue, fell free to reach out.
chenjingcheng
I have barely modified the code, only that the SD model is stored locally.
pip install -r requirements.txt
sholding', 'solver_order', 'variance_type', 'algorithm_type', 'prediction_type', 'timestep_spacing', 'lower_order_final', 'lambda_min_clipped'} was not found in config. Values will be initialized to default values. Traceback (most recent call last): File "/home/aifont/disk4T/aiproject/t2i_font/ClassDiffusion/train_class_diffusion.py", line 1718, in
main(args)
File "/home/aifont/disk4T/aiproject/t2i_font/ClassDiffusion/train_class_diffusion.py", line 1585, in main
images = [
File "/home/aifont/disk4T/aiproject/t2i_font/ClassDiffusion/train_class_diffusion.py", line 1586, in
pipeline(
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 1000, in call
noise_pred = self.unet(
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(args, kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/diffusers/models/unets/unet_2d_condition.py", line 1135, in forward
emb = self.time_embedding(t_emb, timestep_cond)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, *kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/diffusers/models/embeddings.py", line 376, in forward
sample = self.linear_1(sample)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(args, kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/home/aifont/anaconda3/envs/classdiffusion/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 116, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Half
Steps: 10%|████████ | 50/500 [00:07<01:09, 6.49it/s, loss=1.05, lr=2e-5, recon_loss=0.206, spl=0.845]
Traceback (most recent call last):