search

huawei-noah / AdderNet

Code for paper " AdderNet: Do We Really Need Multiplications in Deep Learning?"
BSD 3-Clause "New" or "Revised" License
954 stars 187 forks source link

AdderNet Upsample #56

Closed jdsjfi closed 1 year ago

jdsjfi commented 3 years ago

Hello, I want to realize the Transposed Adder layer according to the relationship between convolution and transposed convolution, but from the result I found that the image it generates has very obvious checkerboard artifacts. I think that it is because the variance of the output of AdderNet is much larger than CNN. So I want to know if you could give me some suggestions to solve it? Thanks very much!!!

HantingChen commented 3 years ago

We add BN layer after adder layer to solve this problem in our paper.

jdsjfi commented 3 years ago

I also add BN layer after each Transposed Adder layer what I mentioned above and the feature maps it generated also have obvious checkerboard artifacts. Could you help me solve it? Thanks a lot!

HantingChen commented 3 years ago

How if you use the transposed conv layer?

jdsjfi commented 3 years ago

When I use transposed convolution, it also has a little checkerboard artifacts, but it is not very obvious if you do not observe it carefully.

HantingChen commented 3 years ago

I think this may caused by some optimization problems. Maybe you can try to adjust the hyper-parameters such as learning rate.

ckyrkou commented 3 years ago

Perhaps you can try using upsampling and then apply normal conv2d/adder2d similar to what is discussed here https://distill.pub/2016/deconv-checkerboard/