search

ml-explore / mlx-examples

Examples in the MLX framework
MIT License
6.28k stars 896 forks source link

Use other flux-based model files? #1116

Open puppyapple opened 6 days ago

puppyapple commented 6 days ago

Thanks for the great work! Recently I see new models like shuttleai/shuttle-3-diffusion · Hugging Face, which is finetuned on flux-schnell. Is there any way to use mflux on these models? I tried with replacing the schnell safetensor's file with shuttle 3's in ~/.cache/huggingface/hub, and ran txt2image.py. This gave me the following errors:

ValueError: Received parameters not in model: model.diffusion_model.single_blocks.22.modulation.lin.bias model.diffusion_model.single_blocks.21.linear2.bias model.diffusion_model.single_blocks.5.modulation.lin.bias model.diffusion_model.double_blocks.14.img_attn.qkv.bias……
angeloskath commented 5 days ago

Pushed a fix at #1117. It seems that the checkpoint is not quite compatible and I think the diffusers library loads the model from the transformers folder but the fix should be easier to use since you can just download using huggingface-cli and then export FLUX_SCNHELL=path/to/safetensors and use any model you want.