search

sjtuplayer / anomalydiffusion

[AAAI 2024] AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model
MIT License
114 stars 14 forks source link

Which parts of the training process are frozen? #51

Open bat1115 opened 2 months ago

bat1115 commented 2 months ago

image image image Are the Denoised Unet of LDM and BERTEmbedder all trained? What is the use of configure_optimizers?

sjtuplayer commented 2 months ago

No, both the Denoised Unet of LDM and BERTEmbedder are not trained. The codebase is borrowed from textual inversion, so there are some other options in configure_optimizers. But only the first if branch is valid in our code.

bat1115 commented 1 month ago

No, both the Denoised Unet of LDM and BERTEmbedder are not trained. The codebase is borrowed from textual inversion, so there are some other options in configure_optimizers. But only the first if branch is valid in our code.

What about configure_opt_embedding( ) and configure_opt_model( )? Are they run in the training process?

bat1115 commented 1 month ago

No, both the Denoised Unet of LDM and BERTEmbedder are not trained. The codebase is borrowed from textual inversion, so there are some other options in configure_optimizers. But only the first if branch is valid in our code.

What about configure_opt_embedding( ) and configure_opt_model( )? Are they run in the training process?

My question is how does BERTEmbedder guarantee the effect of z,position = self.transformer(tokens, return_embeddings=True, **kwargs) if the parameters are not updated? Because self.transformer = TransformerWrapper() is not a pre-trained model.