YuxinWenRick
commented
8 months ago Hi, not sure why there's a data type conflict. Did you load the reference model with float16?
Can you try to modify this line to mg_batch = torch.concatenate(img_batch).to(device).to(torch.float16)?
Traceback (most recent call last): File "run_tree_ring_watermark.py", line 218, in
main(args)
File "run_tree_ring_watermark.py", line 131, in main
sims = measure_similarity([orig_image_no_w, orig_image_w], current_prompt, ref_model, ref_clip_preprocess, ref_tokenizer, device)
File "/home/aistudio/tree-ring-watermark/optim_utils.py", line 88, in measure_similarity
image_features = model.encode_image(img_batch)
File "/home/aistudio/tree-ring-watermark/open_clip/model.py", line 213, in encode_image
features = self.visual(image)
File "/home/aistudio/work/conda3/envs/tree/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(input, kwargs)
File "/home/aistudio/tree-ring-watermark/open_clip/transformer.py", line 482, in forward
x = self.ln_pre(x)
File "/home/aistudio/work/conda3/envs/tree/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(input, kwargs)
File "/home/aistudio/tree-ring-watermark/open_clip/transformer.py", line 28, in forward
x = F.layer_norm(x, self.normalized_shape, self.weight, self.bias, self.eps)
File "/home/aistudio/work/*conda3/envs/tree/lib/python3.8/site-packages/torch/nn/functional.py", line 2487, in layer_norm
return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: expected scalar type Half but found Float