Mask prompt or mask method

[luoshuiyue]

Hello, may I ask what is your method of making mask? If it is extracted based on sam model, may I ask what specific prompt can be provided to extract the bread and the plane under the bread?

[turandai]

Hi, we make the mask from the depth image and depth range provided in the original blendedMVS dataset.

[luoshuiyue]

Thanks a lot! I have another question to ask. Compared with the original Gaussian Splatting method, Gaussian surfels takes up a lot of GPU memory during training. What are the reasons or possible reasons for the large memory occupation, and can you provide some ideas to solve the large GPU memory occupation.

[turandai]

From my perspective, possible reasons are: 1) the normal priors take up additional memotry, 2) gaussian surfels are back-face-culled during optimization and rendering, so it usually need more points than 3dgs to represent a same scene, 3) regulization terms generate addiitional gradient graph that also take up memory. You may try to deal with the these points. Btw, I remember the increase was not very much in my experiments, I am not sure if there is other problems.

[luoshuiyue]

Thank you for your reply. I also want to ask, what is the function of idr format pose data? I found that the pose data input can be colmap format, before-idr-processed format, and after-idr-processed format. I found that sometimes these three data can be trained, but for different dataset, I sometimes run the code directly without checking, finding the idr data processing script will report errors. And the train.py script will also report errors on pose data with idr format.

[turandai]

It seems there is some problem when running IDR code? You may check if all of your input images and cameras are correct. I feel that if the IDR code is properly done, the training process should be right.