search

rohitgandikota / erasing

Erasing Concepts from Diffusion Models
https://erasing.baulab.info
MIT License
559 stars 34 forks source link

A question about how to conduct customized training #21

Open Brembles opened 1 year ago

Brembles commented 1 year ago

Your work is fantastic. If I want to train a model to specifically remove a concept similar to Van Gogh, how many epochs would be required ？

rohitgandikota commented 1 year ago

Thanks for your interest! I believe somewhere around 500-1000 epochs should do the job. If you are using the source code from our demo (diffusers version), then I would suggest 200 epochs. If you are interested in our demo, it is linked on our project website.

Brembles commented 1 year ago

Thank you very much for your guidance! When I was training using your model, I only used Van Gogh as the prompt, and also set words as the prompt. The learning rate was set to 1e-5 and I only trained the xattn. I trained for 200 epochs at a time, and once for 500 epochs, but the results were not as good as the checkpoint you provided. How did you do it?

rohitgandikota commented 1 year ago

We trained it exactly the way you described. When you say "results are not as good" - how do you mean? Could you share some results?

Brembles commented 1 year ago

38_0 38_0 What I mean is that my checkpoint doesn't completely remove the concept of "Van Gogh" under certain prompts. Could you share your training parameters with me?

rohitgandikota commented 1 year ago

Aah, so the ESD model is not loaded properly! clearly they look very close to each other. We used the default parameters from the training script we provided. I would look at the file name and it is being loaded properly. In the evaluation script before you do .load() please check if the file it is trying to load exists. by doing os.path.exists(model_path) .

Brembles commented 1 year ago

The checkpoint I published is the one I trained. Your checkpoint performs very well, so I think my model loading is correct. I don't understand where I went wrong with my training parameters. Currently, I have set train-epoch=500 and lr=1×10^-5. Could you tell me your training parameters?

rohitgandikota commented 1 year ago

As I mentioned, we use the default parameters in the training script - LR 1e-5 and 1000 epochs

Brembles commented 1 year ago

Thank you for your guidance, I will try again!

Brembles commented 1 year ago

I retrained the model but the results are still not good. Maybe you could give me your email address, so I can contact you through email.