GaParmar / img2img-turbo

One-step image-to-image with Stable Diffusion turbo: sketch2image, day2night, and more
MIT License
1.69k stars 190 forks source link

Questions About Training the Sunny2Rainy Model #73

Open HaseongJung opened 3 months ago

HaseongJung commented 3 months ago

Hi,

I'm interested in training the sunny2rainy model and I have a few questions regarding your experiments:

How many data samples did you use for training? How many steps did you train the model for? How long did it take to complete the training? How did you construct the prompts? Thank you for your help!

GaParmar commented 3 months ago

Hi,

We used different data samples for the different tasks (based on the available datasets). You can typically see reasonable results within a few thousand steps. Monitoring the evaluation metrics like FID and DINO-Structure-Similarity would help find the optimal number of steps.

To construct the prompts, we try to describe the domains. For the sunny2rainy model, you could perhaps try prompts like "a picture of a driving scene on a bright sunny day" and "a picture of a driving scene on a rainy day".

-Gaurav

zhangsngood commented 1 month ago

Hi,

I'm interested in training the sunny2rainy model and I have a few questions regarding your experiments:

How many data samples did you use for training? How many steps did you train the model for? How long did it take to complete the training? How did you construct the prompts? Thank you for your help! @HaseongJung Hello, have you successfully trained the "sunny2rainy" model? I am currently working on this as well, using my own dataset for training. However, the rainy style in the resulting rainy images, such as the road surface, is not very pronounced. Have you encountered the same issue? Do you have any suggestions regarding this?

HaseongJung commented 1 month ago

@zhangsngood

In my project, the image transformation performed well during the training process. The performance matrix also came out well: fid: 56, dino: 0.003. However, the results inferred using this model checkpoint are not good.

Sorry I can't give you any suggestion 😂 That's a challenge we have to solve.