Add Image to Image option

filipstrand / mflux

A MLX port of FLUX based on the Huggingface Diffusers implementation.

656 stars 48 forks source link

Add Image to Image option #16

Open Xuzzo opened 3 weeks ago

Xuzzo commented 3 weeks ago

Hi, this PR:

introduces the possibility to modify another image through flux1
fixes a bug with width <-> height (they were swapped in config init)
adds main_img2img for generation. Note: not sure this is needed, we could probably unify cli and keep only one main

filipstrand commented 3 weeks ago

@Xuzzo Again, thanks for the contribution. Very nice that you spotted and fixed the height/width mistake.

Regarding this feature, do you have a good example of it working? I only tried it quickly with the schnell model for 4 steps, and did not get any special result from it (I basically got back the input image with minor variations). Maybe it might not work as well with the distilled schnell model??

Xuzzo commented 3 weeks ago

Using the schnell model I found it is quite important to tune the strength parameter. For example:

Original image:

CLI: python main_img2img.py --prompt "Luxury food photograph" --steps 10 --seed 2 --base-image image.png --strength 0.3

Result: image(37)

I found the dev to be a bit less sensitive to strength.

filipstrand commented 3 weeks ago

Using the schnell model I found it is quite important to tune the strength parameter. For example:

Original image:

CLI: python main_img2img.py --prompt "Luxury food photograph" --steps 10 --seed 2 --base-image image.png --strength 0.3

Result:

I found the dev to be a bit less sensitive to strength.

Oh great, this is a very nice example! I think we should add this to the readme.

filipstrand commented 3 weeks ago

Btw, I have looked at your PR, and will share some comments later today. Thanks for the patience :)