Closed Magauiya closed 3 years ago
1a. I believe the PSNR gap is due to two factors. First, the images in BurstSR have smaller intensity values in general, compared to Synthetic Dataset. This leads to higher PSNR values. Secondly, due to spatial and color mis-alignment issue, we align the network prediction to the ground truth when computing PSNR on the BurstSR dataset. This may also lead to increased PSNR value.
b. We have seen that directly applying network trained on only synthetic data to images from BurstSR leads to poor results which indicates that there is indeed a significant distribution shift.
c. The model was trained to operate on large bursts (burst size 8). Thus when using very small bursts (burst-2), its performance can be poor since the benefits of merging information from multiple frames can be outweighed by the harms caused by e.g. small mis-alignments. If we train a network using smaller bursts (burst size 2), then we observed that it obtains similar or slightly better results compared to the single image baseline, which using burst with 2 images.
d. Yes, for the user study we converted images to sRGB.
e. No, the network was trained using 56x56x4 (last channel contains RGGB values) patches.
2a. We will post the challenge report on arxiv next week.
b. Yes, we used the same test set.
Regards Goutam
1a. I believe the PSNR gap is due to two factors. First, the images in BurstSR have smaller intensity values in general, compared to Synthetic Dataset. This leads to higher PSNR values. Secondly, due to spatial and color mis-alignment issue, we align the network prediction to the ground truth when computing PSNR on the BurstSR dataset. This may also lead to increased PSNR value.
b. We have seen that directly applying network trained on only synthetic data to images from BurstSR leads to poor results which indicates that there is indeed a significant distribution shift.
c. The model was trained to operate on large bursts (burst size 8). Thus when using very small bursts (burst-2), its performance can be poor since the benefits of merging information from multiple frames can be outweighed by the harms caused by e.g. small mis-alignments. If we train a network using smaller bursts (burst size 2), then we observed that it obtains similar or slightly better results compared to the single image baseline, which using burst with 2 images.
d. Yes, for the user study we converted images to sRGB.
e. No, the network was trained using 56x56x4 (last channel contains RGGB values) patches.
2a. We will post the challenge report on arxiv next week.
b. Yes, we used the same test set.
Regards Goutam
Thank you for the detailed answers!
I am a bit curious about the network complexity (# of parameters, macs), because your proposed model with single image input works an order of magnitude better than our model on 14 frames for Track 2.
You're welcome!
That's strange. Including the parameters of our alignment network (PWCNet), our model had 13011237 parameters. Excluding the alignment network, the number of parameters is 3636963. Detailed network architecture, including number of residual blocks, channel dimensions etc is provided in our supplementary.
How is the performance on the synthetic set? I would suggest first ensuring that your model works well on the synthetic dataset.
Regards Goutam
You're welcome!
That's strange. Including the parameters of our alignment network (PWCNet), our model had 13011237 parameters. Excluding the alignment network, the number of parameters is 3636963. Detailed network architecture, including number of residual blocks, channel dimensions etc is provided in our supplementary.
How is the performance on the synthetic set? I would suggest first ensuring that your model works well on the synthetic dataset.
Regards Goutam
The PSNR on synthetic validation dataset is ~43dB and on test is ~46-47dB. We have used 14 frames. The model is huge.
Do you have plans to release your implementation?
Sincerely, Magauiya
Dear organizers and authors,
I would appreciate if you clarify following questions:
Thank you!