Question About Super-Splatting

[Rashfu]

The original text says, Specifically, the 2D Gaussian projections in screen space are super-resolved to synthesize HR novel views. But what does it mean to "super-resolve" 2D Gaussians? I'm not clear on how this is actually implemented.

And I look forward to the updated README.md, which will tell us how to use your code.

Thanks in advance!

[XiangFeng66]

Thank you for your attention. First of all, 3DGS splashes 2DGS onto the low-resolution rendering image during low-resolution training, and SRGS directly splashes out the high-resolution rendering image. This is called super-resolve. Secondly, the code of SRGS is very simple, and there are few changes in 3DGS. It is easy to understand if you read it carefully.

[Rashfu]

I see what you mean. Thanks for the explanation.

[XiangFeng66]

Hello, have you run my code? If you reproduce some results, please contact me. I have some questions for you. Thank you.

[Rashfu]

Hi @XiangFeng66, I tried reproducing your results on the Synthetic NeRF, Tanks & Tamples datasets and Mip-NeRF 360 Dataset., but I failed. （single RTX 3090）

• For Synthetic NeRF dataset, the code gave me the error like this:

  File "/home/dell/SRGS/scene/dataset_readers.py", line 218, in readCamerasFromTransforms
  image_path=image_path, image_name=image_name, width=image.size[0], height=image.size[1]))
  TypeError: __new__() missing 1 required positional argument: 'image_ori'

  I guess this is the bug for dataset preparation.

• For Tanks & Tamples dataset, I don’t know the specific data you chose, and the raw data likely needs to be processed through COLMAP, so I decided to give up.

• For Mip-NeRF 360 Dataset. it shows "CUDA out of memory."

As for the code, I'm curious why resolution_ori is fixed to be multiplied by 4 in the loadCam function here. If I want to conduct a 2x super-resolution experiment, should I change it to 2? If that's the case, there might be other parts of the code where the factor is also fixed to 4 and would need to be modified.

[XiangFeng66]

Yes，for the Tank dataset, it needs to be processed by colmap. Regarding the blender dataset, you need to refer to the colmap reading method to modify it, because this code is not organized by me. If you want to experiment with X2, you need to modify the Loadcam function and the Camera class. I think it is very simple.

[Rashfu]

Thanks for your reply. I will try to reproduce the results.

• However, for the Mip-NeRF 360 dataset, why does the code encounter an “out of memory” issue (the preprocessing of super-resolution data takes nearly half an hour)?
• And Why was only loss_sr used in the final loss calculation, without incorporating the sub-pixel constraints mentioned in the paper?

[XiangFeng66]

Sorry about the OOM issue, I think you should use RTX4090. The two-dimensional model processing data must be inside the code to prevent distortion of the pictures during the data storage process, so Swinir needs to reason about each picture, which will consume a little time. For the loss problem, I use loss_sr for ablation experiments. You can add the following loss_lr to reproduce the results of our paper.