search

lllyasviel / stable-diffusion-webui-forge

GNU Affero General Public License v3.0
7.45k stars 722 forks source link

Flux dev FP16 or BF16 quant compatibility #1142

Open KSimulation opened 1 month ago

KSimulation commented 1 month ago

I tried to quantize the model into BF16 and FP16 to preserve a bit more precision than NF4/FP8 while still running fast on data center cards. However, when I tried to hack the way to get it to run in ForgeUI, I can only get it to generate nonsense donut pictures (just like the 1st image of black hole).

Any ideas of where I need to change/hack to get it to work? Feels like there is a data scramble happening somewhere

@lllyasviel hopefully we can get this as a native feature : ]

Thanks XD K.

KSimulation commented 1 month ago

Seems already fixed: https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/1050