search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
23.97k stars 4.93k forks source link

[Tracker] modularize inferencing during and after training in the example scripts #6545

Open sayakpaul opened 5 months ago

sayakpaul commented 5 months ago

We provide support for running validation inference during and after training in our officially maintained training examples. This is very helpful to keep track of the training progress.

We could modularize some bits in the example to reduce the LoC.

The train_lcm_distill_lora_sdxl.py script already does this:

https://github.com/huggingface/diffusers/blob/33d2b5b08764d2579dd4ea555821b8986cd2dcf9/examples/consistency_distillation/train_lcm_distill_lora_sdxl.py#L100

It would be nice to follow something similar for the rest of the scripts too. Here's a handy list of the scripts where we'd like to incorporate this change:

Feel free to comment here if you're interested.

Also, when opening PRs, please target one example at a time and please tag me in the PRs.

charchit7 commented 5 months ago

Hey @sayakpaul for text-to-image, SD log-validation is present : https://github.com/huggingface/diffusers/blob/33d2b5b08764d2579dd4ea555821b8986cd2dcf9/examples/text_to_image/train_text_to_image.py#L141

haven't gone through the code, will check more if we need to modify it.

sayakpaul commented 5 months ago

Ideally, this should be delegated to log_validation() too: https://github.com/huggingface/diffusers/blob/33d2b5b08764d2579dd4ea555821b8986cd2dcf9/examples/text_to_image/train_text_to_image.py#L1047

As done in train_lcm_distill_lora_sdxl.py.

charchit7 commented 5 months ago

Got it. @sayakpaul I missed that part. Thanks!

neeeemo commented 5 months ago

Hey @sayakpaul, @charchit7 I'm interested in incorporating the change to InstructPix2Pix SD

coolyashas commented 5 months ago

Hey @sayakpaul, @charchit7 I'm interested in incorporating the change to InstructPix2Pix SD

Hey @sang-k, I have been looking into InstructPix2Pix SD too. Would be happy to collaborate if necessary.

neeeemo commented 5 months ago

@coolyashas Oh, I see. I will look into InstructPix2Pix SDXL then!

coolyashas commented 5 months ago

Working on InstructPix2Pix SD