ml-explore / mlx-examples

Examples in the MLX framework
MIT License
6.3k stars 898 forks source link

Add FLUX finetuning #1028

Closed angeloskath closed 1 month ago

angeloskath commented 1 month ago

This adds a dreambooth example for FLUX-dev finetuning. It still needs a README and possibly some more cleaning but it works well for the dev model. The following is a list of things to do in the future

Distributed training scales almost perfectly, most of my training was with 8 nodes.

akkikumar72 commented 1 month ago

Awesome work 🎊 Will be nice to have a readme for this, so we can try/test it in weekend

angeloskath commented 1 month ago

@akkikumar72 I totally agree. I am AFK for a few hours but I will write the README and get it merged tonight (PST).

nattaylor commented 1 month ago

This is amazing. Any idea when you might have QLoRA implemented? :)