train issue

[goodman-gao]

hi, I trained the 'finetune LDM Decoder. py‘ and the number of steps was increased, but the model did not learn during the training process. The loss has not decreased，I printed the watermark for the training image and the watermark for the validation image, but it was much different from the randomly generated watermark. I would like to ask for the reason and how to set the parameters in the code. thanks

[pierrefdz]

Hi, (and sorry for the late reply!!). Could you share more details? Fine-tuning arguments, logs, etc.?

[goodman-gao]

Thank you. I trained the 'finetune LDM Decoder. py‘ and during the verification process, we obtained the results from your paper. But I found that the model I obtained is only applicable to the randomly generated watermark encoding during my training, and not to other watermark encodings. I would like to confirm with you if this is the case.

[pierrefdz]

I think I might be missing something. The goal of Stable Signature is to modify the weights such that generated images carry a certain message, fixed in advance. The modification is done via the fine-tuning of the LDM's decoder. At the end of this, all generated images should carry the same message. If you want them to carry another message, you need to re-do the fine-tuning with another message.

[goodman-gao]

Thank you, I understand, but now I would like to train a model that can adapt to more watermarks. Is there any method available？

[pierrefdz]

Hi,

You can try to look at methods like

• https://arxiv.org/abs/2306.04744, if you want to distribute watermarked models
• https://arxiv.org/abs/2304.03400, if you want to distribute watermarked images

[pierrefdz]

Closing since no more activity, don't hesitate to re-open