is “Instance Normalization” used in this paper?

sniklaus / softmax-splatting

an implementation of softmax splatting for differentiable forward warping using PyTorch

466 stars 58 forks source link

is “Instance Normalization” used in this paper? #15

Closed Hsveh closed 4 years ago

Hsveh commented 4 years ago

Very impressive work! But I have some questions when I finished reading the paper. You mentioned that you apply "Instance Normalization" to ctxsyn in the paper "Context-aware Synthesis for Video Frame Interpolation", and I have 2 questions about the details. (1) is “Instance Normalization” also used in this paper? (2) if (1) is true, how do you jointly normalize the two input frames? (2) if (1) is true, are the inputs of the feature extractor normalize?

sniklaus commented 4 years ago

Thank you for your kind words!

(1) Yes. (2a) By computing the statistics jointly for the two images, instead of separately. (2b) Yes.

Fairydetail commented 3 years ago

Hello，as the paper "Context-aware Synthesis for Video Frame Interpolation" described below, I have several questions.@sniklaus (1) is “Instance Normalization” used in the inputs of the optical ﬂow estimator? (2) are the inputs of the Z metric normalized? (3) if (1) is yes. Since the distribution of input data has changed after instance normalization, is it not suitable for the optical flow estimation module to load the pretrained params? (4) if (1) is yes. Is the normalization be reversed on "the resulting output from the synthesis network" in this paper?

sniklaus commented 3 years ago

(1) Yes. (2) No, we do not normalize the features used to compute the importance metric. (3) We have trained our own PWC-Net with the instance normalization in place. (4) Yes, it has to.