issue when running inference with dtype mismatch

[ShristiDasBiswas]

/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True. warnings.warn( vae/diffusion_pytorch_model.safetensors not found Loading pipeline components...: 86%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 6/7 [00:02<00:00, 1.98it/s]/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead. warnings.warn( Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:02<00:00, 2.41it/s] 0%| | 0/50 [00:00<?, ?it/s] Traceback (most recent call last): File "/mnt/efs-shared/sdasbisw/differential-diffusion/promptdiff.py", line 32, in image = pipe("a tortoise", num_inference_steps=50, generator=generator, image_pair=[image_a,image_b], image=query).images[0] File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/mnt/efs-shared/sdasbisw/differential-diffusion/Prompt-Diffusion/pipeline_prompt_diffusion.py", line 1237, in call down_block_res_samples, mid_block_res_sample = self.controlnet( File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(args, kwargs) File "/mnt/efs-shared/sdasbisw/differential-diffusion/Prompt-Diffusion/promptdiffusioncontrolnet.py", line 279, in forward emb = self.time_embedding(t_emb, timestep_cond) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, *kwargs) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/diffusers/models/embeddings.py", line 227, in forward sample = self.linear_1(sample) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/home/ubuntu/anaconda3/envs/diffdiff/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 116, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Half