Closed ghost closed 1 year ago
For FP32 they aint different
For model compatibility reasons, we perform normalization operation at fp32: https://github.com/liuliu/ccv/blob/unstable/lib/nnc/cmd/norm/mps/ccv_nnc_layer_norm_mps.m#L63 I think maple-diffusion just do it at fp16: https://github.com/madebyollin/maple-diffusion/blob/main/maple-diffusion/MapleDiffusion.swift#L153
What compatibility reasons?
Thanks, is there any other differnce, apart from groupnorm and layernorm?
Right now, in swift-diffusion the FP16 output is very diffrent from swift-diffusion FP32 output. Although Maple's FP16 output is very close to Swift-diffusions FP32 output.
That is causing quality drop in swift diffusion for fp16.
There is some other difference too. i did layernorm and groupnorm at all fp16, still i am seeing a big difference.
Thanks!
Try
DynamicGraph.flags = [.disableMixedMPSGEMM, .disableMixedMPSSoftMax]
See if you can get anywhere closer. If not, there might be something more.
Thanks a lot, it gets much closer. Btw this will fix the inpainting issue you have ( mentioned in the readme) .
now normal inpainting should work.
What does MixedMPSGEMM and MixedMPSSoftMax do? Is there a bug in it?
https://github.com/liuliu/ccv/blob/unstable/lib/nnc/cmd/softmax/mps/ccv_nnc_softmax_mps.m#L18
https://github.com/liuliu/ccv/blob/unstable/lib/nnc/cmd/blas/mps/ccv_nnc_gemm_mps.m#L123
You can dig deeper to see if particular one caused issues.
Also, are you doing this with SD v1.x models or SD v2.x models?
1.x
v2.x requires to disable MixedMPSGEMM but v1.x I don't find a reason to (I also didn't do numerical accuracy analysis like you did, just eyeballing end-result).
I input the same exact tensors to Unet of both swift diffusin and maple diffuion, but outputs are little diffrent. The diffrence for each element is from 0.01 to 0.06.
Why is that so? I am using the NWHC branch