About the format of the provided raw images?

[Askiry]

hi, I wonder why the raw images is provided as .png format?

And in the real-world track dataset, how can they be visually displayed to check the degree of spatial misalignment?

[goutamgmb]

We saved the RAW images in 4 channel .png format to save disk space. Please refer to scripts/test_burstsr_dataset.py for an example on how to visualize the RAW images from the BurstSR dataset. While the script only shows the first image in the burst, you can easily modify it to show multiple burst images, to check spatial misalignment.

[Askiry]

OK, thanks. And I also wonder, in real mobilephone SR (zoom) application, if the multi-frame SR is done in the RGB/YUV domain is more feasible than in the RAW domain? Because in my opinion, if SR is done in RAW domain, the following modules in the ISP pipline must operate on larger image size. This will cause lots of computation burden.

[goutamgmb]

Yes, doing SR in RAW domain would lead to additional computation burden on the post-processing steps. However, doing SR in RAW domain instead of RGB/YUV domain has also been show to provide better results (https://arxiv.org/abs/1905.05169). Thus, if computationally feasible, it is preferable to do the super-resolution in RAW domain itself.

[Askiry]

Thanks for your reply. I wonder, if doing SR in RGB/YUV domain, whether the related RGB/YUV datasets including the synthetic and real ones can be provided or generated from your provided RAW datasets?

[goutamgmb]

Generating synthetic RGB/YUV burst dataset should be simple by modifying the synthetic_burst_generation.py script. For example, you can remove this line to get data in sensor RGB space, instead of RAW space.

For the real world data, you can use existing pipelines (e.g. AHD for demosaicking, BM3D for denoising) to first convert the RAW data to RGB/YUV space, and then perform super-resolution.