search

rinongal / textual_inversion

MIT License
2.87k stars 278 forks source link

Different init_words have same loss_epoch curve #151

Closed Sutongtong233 closed 1 year ago

Sutongtong233 commented 1 year ago
image

As shown in figure, we use thin_bird daatset, and the run scripts are the same, except the init_word: "bird"/"toy". However, when I check the log, their loss_epoch curve are the same. How does init_word work?

hkeziah commented 1 year ago

Unsubscribe

On May 8, 2023, at 9:01 AM, TT @.***> wrote:

https://user-images.githubusercontent.com/56153611/236830314-1f8322bd-5f85-42f6-881c-6f19548764da.pngAs shown in figure, we use thin_bird daatset, and the run scripts are the same, except the init_word: "bird"/"toy". However, when I check the log, their loss_epoch curve are the same. How does init_word work? — Reply to this email directly, view it on GitHub https://github.com/rinongal/textual_inversion/issues/151, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFCDDSDJZITKCIVH6GBJLDXFDVBDANCNFSM6AAAAAAXZ5HE7Y. You are receiving this because you are subscribed to this thread.

rinongal commented 1 year ago

Hi @Sutongtong233 The loss itself isn't very informative as is because it is highly dominated by randomness in training. The sampled images, sampled timesteps and sampled noises all have a much greater impact on the loss values than the actual decrease in training.

Please see https://arxiv.org/abs/2302.04841 for a more in-depth discussion and a way to 'fix' this if you want to have an informative loss.

Beyond that - init_word should be replacing the embedding that the optimization begins from. The best way to check if it is actually having an effect is to open your logged images and make sure they aren't the same between the two runs.

Sutongtong233 commented 1 year ago

Thanks!