Closed wilhelmw201 closed 10 months ago
w_a = torch.moveaxis(w_a, axis, 0).reshape((n, -1)).to("cpu").to(torch.float32)
w_b = torch.moveaxis(w_b, axis, 0).reshape((n, -1)).T.to("cpu").to(torch.float32)
A += torch.matmul(w_a, w_b).cpu()
@wilhelmw201 Missed this sorry. Should work now.
after the first iteration i get the following error, when i use float32 mode to merge models
Traceback (most recent call last): File "E:\stablediffu\stable-diffusion-webui\models\Stable-diffusion\Merge-Stable-Diffusion-models-without-distortion-main\SD_rebasin_merge.py", line 65, in
first_permutation, y = weight_matching(permutation_spec, flatten_params(model_a), theta_0, usefp16=args.usefp16)
File "E:\stablediffu\stable-diffusion-webui\models\Stable-diffusion\Merge-Stable-Diffusion-models-without-distortion-main\weight_matching.py", line 849, in weight_matching
A += torch.matmul(w_a, w_b).cpu()
RuntimeError: expected scalar type Float but found Half