Questions about Ablation Study Procedures

[HsiehTPE]

Hello, thank you for your work on the paper and code. I am also interested in replicating the ablation study section, but I have some questions about the specific methods.

Regarding the Gaussian Initialization operation, is it possible to understand the use of COLMAP and EndoNeRF for initialization as a difference in dataset reading? That is, the discrimination of scene_info in

scene/init.py, lines 39~42,

to the sceneLoadTypeCallbacks operation in

scene/dataset_readers.py.

Also, how should the random initialization method be implemented, and is this part reflected in the code?

Besides, for the Gaussian Tracking part, is there any code implementation available that uses MLP replacement?

Thank you very much for your attention and response. Wish you have a good day : )

[yifliu3]

Hi, thanks for your attention.

For Gaussian initialization, we typically have two ways, initialize from COLMAP or the designed HGI module in the paper. To use HGI module, you can directly use the EndoNeRF dataloader where we have implemented the initialization algorithm. For the COLMAP initialization, we need to first run the COLMAP algorithm to get various files including sparse points and camera poses, etc., as stated by the original 3DGS: https://github.com/graphdeco-inria/gaussian-splatting, and then use the COLMAP dataloader to load the data.

For Gaussian tracking, you can disable the Hexplane(grid) and change the MLP to 8 layers to make a fair comparison. Hope this can solve your problem :)

[HsiehTPE]

Dear devs, Firstly, thanks for your reply and also sorry for my late response.

Following your instruction, I have prepared the files for the COLMAP initialization. And here is how I place them: However, once I started the training process, an error occured: The error is

AttributeError: 'CameraInfo' object has no attribute 'FoVx'

After reading the code, I guess the possible reason as follows: While running into

File "train.py", line 115, in scene_reconstruction render_pkg = render(viewpoint_cam, gaussians, pipe, background, stage=stage),

the class type of viewpoint_cam is CameraInfo in

dataset_readers.py.

But the class should be the type of Camera in

cameras.py.

That is, the COLMAP dataset is not processed properly. So I wonder if I found the correct cause of the error and how to fix it, or I did not pre-processed the COLMAP correctly.

Again, thank you very much for your attention to my question, and thank you very much for your answer. Looking forward to your reply, wish you have a good day o(￣▽￣)ブ

[yifliu3]

Hi,

I just checked the codes about the COLMAP part, which is in fact inherited from 4DGS, so there are incompatibilities. In particular, we use the Camera class instead of CameraInfo, and you can just replace the CameraInfo class with the Camera, and make sure the keywords are matched. Apart from that, training and testing split should also be matched with the endoloader.

I will update the code later. Thanks for your attention.

[HsiehTPE]

Hello,

Thanks for your patience and reply! I will try again later. And I will reply this post once I make it work.

Thanks again : )