Awesome

[nitinmukesh]

Thanks for sharing with us, works pretty good.

I tried the fp16 version of models and made code changes in frompretained (wherever necessary by checking the fp16 version model in https://huggingface.co/MeissonFlow/Meissonic/tree/main) torch_dtype=torch.float16, variant="fp16",

But it comes with error. Anyway to use fp16 version as it will consume even less memory

[Naugustogi]

Just convert it to Safetensors and upload it to Huggingface. Its just an diffusers.

[viiika]

Apologies for the delayed response. A simple approach to using the fp16 model is to first load the fp32 model with from_pretrained, then convert it to fp16 using .half(). Afterward, you can assemble the components in the pipeline and move them to the appropriate device. Currently, directly loading the text encoder with variant="fp16" has some unresolved issues, which we’re working to address.

[nitinmukesh]

@viiika

Thanks for your response. Looking forward for the fix.

[nitinmukesh]

@viiika

Just asking as I was thinking of creating Gradio Ui for FP8, is FP16 fixed or is it .half().

Performance Benchmarks

Precision (Steps=64, Resolution=1024x1024) | Batch Size=1 (Avg. Time) | Memory Usage -- | -- | -- FP32 | 13.32s | 12GB FP16 | 12.35s | 9.5GB FP8 | 12.93s | 8.7GB

[viiika]

I'm sorry, but I don't fully understand your question. As of now, there are no updates regarding the fp16 issue.

[nitinmukesh]

Got it, thanks.

[viiika]

python inference_fp16.py

This script supports fp16 inference. Just set torch_dtype and do not set variant.