Can I use this tool to generate 3D Gaussian splatting datasets?

[wangjinhoon]

"Hello, I'm curious if this tool can be used to generate 3D Gaussian splatting datasets. Thank you!"

[maximeraafat]

Hi @wangjinhoon, thanks for your question!

I haven't had the chance to play much with Gaussian splatting yet, but as far as I know they also support NeRF datasets. According to the readNerfSyntheticInfo function in the scene/dataset_readers.py file of the official Gaussian splatting repository, you can directly use the same transforms_train.json and transforms_test.json files:

https://github.com/graphdeco-inria/gaussian-splatting/blob/2eee0e26d2d5fd00ec462df47752223952f6bf4e/scene/dataset_readers.py#L221-L225

Notice though that Gaussian splats typically get initialised with an optional point cloud estimate of the scene (obtained via colmap) for faster convergence. In the case of BlenderNeRF, there is no direct or easy way to get a point cloud of the geometry. You could circumvent this by sampling points from your object in question using geometry nodes, extract and save the points into a points3d.ply file, but this might not work so well with very crowded scenes. The Gaussian splatting code loading the optional point cloud is also below:

https://github.com/graphdeco-inria/gaussian-splatting/blob/2eee0e26d2d5fd00ec462df47752223952f6bf4e/scene/dataset_readers.py#L233-L244

[wangjinhoon]

Thank you for providing the good tool and for your response!"

[jiangyijin]

Thank you for providing the good tool and for your response! Hello, do you realize the production of 3D Gaussian splatting datasets with this plug-in? If there is any other way, can you tell me? Thank you very much!

[wangjinhoon]

Hello jiangyijin I created an image and depth image, camera pose dataset to run SplatTAM, and then adjusted the camera pose data to fit Gaussian splatting.

[jiangyijin]

Hello jiangyijin I created an image and depth image, camera pose dataset to run SplatTAM, and then adjusted the camera pose data to fit Gaussian splatting.

Hello, could you please share the details of the steps and tools used? I've been stuck at this for a long time. Thank you very much!

[wangjinhoon]

I extracted data using the COS of this plugin.

also searched here, on blogs, and on YouTube for extraction data like this https://www.youtube.com/watch?v=C8YuDoU11cg&t=947s

This is the code I used for pose transformation. https://drive.google.com/file/d/1s39cVZcEy_IwXmDghBw9NpECNlv8n0uc/view?usp=sharing

You might need to make some modifications to use it.

I'm sorry I couldn't be more helpful.

[maximeraafat]

Hi @jiangyijin,

I have personally not used Gaussian splatting much yet, but from what I understand according to @wangjinhoon (thank you for the contribution), you might need to apply some changes to the stored camera pose data. You can either edit the camera poses using the method shared by @wangjinhoon, or directly incorporate the necessary changes into the blender_nerf_operator.py file. Hope this helps!