search

aipixel / AEMatter

Another matter.
GNU General Public License v2.0
52 stars 2 forks source link

a question #5

Closed astro-fits closed 9 months ago

astro-fits commented 9 months ago

Hi,

An impressive work. But I have a question. As mentioned in the paper (chapter 4.1.1), " we omit the normalization layers in the stem as they cause internal covariate shift, which hurts the matting performance. "

Can you provide more explanation for that ?

I remember that the internal covariate shift issue could be solved if normalization layers (e.g. batch-normal) are performed during the training.

please find information at 10’51” of the video: 【李宏毅】批量归一化 (Batch Normalization, BN)

https://www.bilibili.com/video/BV1Bz4y1j78f/?spm_id_from=333.1007.tianma.1-1-1.click

Thank you very much.

Windaway commented 9 months ago

Because batchnorm/groupnorm is designed for classification tasks, it aims to stabilize shallow features to expedite the processing by deeper layers during training. On the other hand, image matting relies on the original image information, making it similar to super-resolution tasks. During prediction, it requires access to high-dynamic-range original features. Therefore, using normalization operations would stabilize the features but remove some information, leading to a performance drop. This is an intriguing observation, as our expectations for features differ.

astro-fits commented 9 months ago

Thank you for your reply.