huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
25.67k stars 5.3k forks source link

FLUX.1-dev FP8 Example Code Cleanup #9690

Open hashnimo opened 1 week ago

hashnimo commented 1 week ago

I was looking at the FLUX.1-dev FP8 example code in the documentation and noticed some unnecessary imports and variable declarations, which make it appear longer and cluttered. Here is the shorter version of the code I'm currently using. Please note that I'm not an expert, just trying to make it easier for anyone else looking to run the code.

import torch
from diffusers import FluxTransformer2DModel, FluxPipeline
from transformers import T5EncoderModel
from optimum.quanto import quantize, qfloat8, freeze

bfl_repo = "black-forest-labs/FLUX.1-dev"

transformer = FluxTransformer2DModel.from_single_file("https://huggingface.co/Kijai/flux-fp8/blob/main/flux1-dev-fp8.safetensors", torch_dtype=torch.bfloat16)
quantize(transformer, weights=qfloat8)
freeze(transformer)

text_encoder_2 = T5EncoderModel.from_pretrained(bfl_repo, subfolder="text_encoder_2", torch_dtype=torch.bfloat16)
quantize(text_encoder_2, weights=qfloat8)
freeze(text_encoder_2)

pipe = FluxPipeline.from_pretrained(bfl_repo, transformer=transformer, text_encoder_2=text_encoder_2, torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]

image.save("flux-dev-fp8.png")

Optimizations made: Removed unnecessary code and reduced the line count from 32 to 26.

a-r-r-o-w commented 1 week ago

Thanks for making it leaner! Would you like to open a PR? You can tag @sayakpaul for reviews if you do