diyiiyiii / StyTR-2

StyTr2 : Image Style Transfer with Transformers
364 stars 66 forks source link

content leak #19

Open xyiiinexg3 opened 2 years ago

xyiiinexg3 commented 2 years ago

您好,最近在阅读您的这篇文章,有以下疑惑: In this paper, "typical CNN-based style transfer methods are biased toward content representation by visualizing the content leak of the stylization process", CNN因为ArtFLow中的那3个原因,不是会破坏内容特征吗?为什么说偏向于内容表征呢?

1. Reconstruction error Although an image reconstruction loss [32] or a content loss [20] is used to train the decoder, Li et al. [32] acknowledge that the decoder is far from perfect due to the loss of spatial information brought by the pooling operations in the encoder. Consequently, the accumulated image reconstruction error may gradually disturb the content details and lead to the Content Leak. 2. Biased decoder training Due to Ls, the decoder is trained to trade off between Lc and Ls, rather than trying to reconstruct images perfectly. ...Consequently, the auto-encoder of AdaIN is biased towards rendering more artistic effects, which causes Content Leak. With the increase of the inference rounds, weird artistic patterns gradually appear in the produced results, which indicates that the auto-encoder of AdaIN may memorize image styles in training and bias towards the training styles in inference. 3.Biased style transfer module Since such a patch replacement is irreversible, fc cannot be recovered from fcs, which makes fcs be biased towards style and consequently causes the Content Leak phenomenon. 谢谢。

diyiiyiii commented 1 year ago

感谢关注,“are biased toward content representation”本意是“内容表示是有偏的”,可能表述有歧义 不好意思。