Some questions about the point initialization for the NeRF-synthetic dataset

[yzslab]

Hi, for the NeRF-synthetic dataset, I found that the RGB value of the points are not generated randomly directly, instead, it generate random coefficients first, then the invoke SH2RGB convert to RGB: https://github.com/graphdeco-inria/gaussian-splatting/blob/f11001b46c5c73a0a7d553353c898efd68412abe/scene/dataset_readers.py#L240-L244

The RGB value initialized by this way are identical, as the image show below:

I tried to initialize RGB randomly directly, ranther than via SH2RGB, but it will produce artifacts/floaters around the scene:

...
rgb = np.asarray(np.random.random((num_pts, 3)))
pcd = BasicPointCloud(points=xyz, colors=rgb, normals=np.zeros((num_pts, 3)))
scene_info = SceneInfo(point_cloud=pcd, ...)
...

Comparation (train in black background, render in white background):	via SH2RGB, identical RGB	directly, random RGB

I want to ask:

• Is the identical RGB value of the points your expected result?
• Is the "random Gaussians" mentioned by the paper include random RGB color?:

  we can achieve state-of-the-art results even with random initialization: we start training from 100K uniformly random Gaussians inside a volume that encloses the scene bounds

• Do you have any idea that why random RGB value will produce artifacts/floaters?

[grgkopanas]

I assume this happens because the random colors match the color of the background ideally you would want to randomize consistently both the background color of the rendering and alpha-composite the RGBA gt with the same random color

On Wed, Oct 25, 2023 at 5:46 PM Zhensheng Yuan @.***> wrote:

Hi, for the NeRF-synthetic dataset, I found that the RGB value of the points are not generated randomly directly, instead, it generate random coefficients first, then the invoke SH2RGB convert to RGB: https://github.com/graphdeco-inria/gaussian-splatting/blob/f11001b46c5c73a0a7d553353c898efd68412abe/scene/dataset_readers.py#L240-L244

The RGB value initialized by this way are almost identical, as the image show below: [image: image] https://user-images.githubusercontent.com/564361/278058477-67570d11-0f8b-45e0-b16f-920c87aae9bf.png

I tried to initialize RGB randomly directly, ranther than via SH2RGB, but it will produce artifacts/floaters around the scene:

...rgb = np.asarray(np.random.random((num_pts, 3)))pcd = BasicPointCloud(points=xyz, colors=rgb, normals=np.zeros((num_pts, 3)))scene_info = SceneInfo(point_cloud=pcd, ...) ...

Comparation: via SH2RGB, identical RGB directly, random RGB

https://github.com/graphdeco-inria/gaussian-splatting/assets/564361/29e35ac7-1448-4cd2-a588-76719c60f575 https://github.com/graphdeco-inria/gaussian-splatting/assets/564361/236fe743-4525-4bda-a8cc-097d19593361

I want to ask:

• Is the identical RGB value of the points your expected result?
• Is the "random Gaussians" mentioned by the paper include random RGB color?:

we can achieve state-of-the-art results even with random initialization: we start training from 100K uniformly random Gaussians inside a volume that encloses the scene bounds

• Do you have any idea that why random RGB value will produce artifacts/floaters?

— Reply to this email directly, view it on GitHub https://github.com/graphdeco-inria/gaussian-splatting/issues/387, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGXXYLRNPVAH767D3DM3NDYBEX37AVCNFSM6AAAAAA6PTPNBKVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3DCNZTGM3DKNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[yzslab]

I assume this happens because the random colors match the color of the background ideally you would want to randomize consistently both the background color of the rendering and alpha-composite the RGBA gt with the same random color … On Wed, Oct 25, 2023 at 5:46 PM Zhensheng Yuan @.> wrote: Hi, for the NeRF-synthetic dataset, I found that the RGB value of the points are not generated randomly directly, instead, it generate random coefficients first, then the invoke SH2RGB convert to RGB: https://github.com/graphdeco-inria/gaussian-splatting/blob/f11001b46c5c73a0a7d553353c898efd68412abe/scene/dataset_readers.py#L240-L244 The RGB value initialized by this way are almost identical, as the image show below: [image: image] https://user-images.githubusercontent.com/564361/278058477-67570d11-0f8b-45e0-b16f-920c87aae9bf.png I tried to initialize RGB randomly directly, ranther than via SH2RGB, but it will produce artifacts/floaters around the scene: ...rgb = np.asarray(np.random.random((num_pts, 3)))pcd = BasicPointCloud(points=xyz, colors=rgb, normals=np.zeros((num_pts, 3)))scene_info = SceneInfo(point_cloud=pcd, ...) ... Comparation: via SH2RGB, identical RGB directly, random RGB https://github.com/graphdeco-inria/gaussian-splatting/assets/564361/29e35ac7-1448-4cd2-a588-76719c60f575 https://github.com/graphdeco-inria/gaussian-splatting/assets/564361/236fe743-4525-4bda-a8cc-097d19593361 I want to ask: - Is the identical RGB value of the points your expected result? - Is the "random Gaussians" mentioned by the paper include random RGB color?: we can achieve state-of-the-art results even with random initialization: we start training from 100K uniformly random Gaussians inside a volume that encloses the scene bounds - Do you have any idea that why random RGB value will produce artifacts/floaters? — Reply to this email directly, view it on GitHub <#387>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGXXYLRNPVAH767D3DM3NDYBEX37AVCNFSM6AAAAAA6PTPNBKVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3DCNZTGM3DKNQ . You are receiving this because you are subscribed to this thread.Message ID: @.>

Hi, do you mean that the RGB values of the points are relating to the background color? But I do not found any codes about this.

[qhdqhd]

The author means that the random color should not be a background color such as pure black or pure white.

[yangqinhui0423]

@yzslab Hello, have you solved the confusion? I found that my NeRF-synthetic dataset showed similar black artifacts (or noise) in both two cases (SH2RGB and random RGB) as in the second video you showed.

[yzslab]

@yzslab Hello, have you solved the confusion? I found that my NeRF-synthetic dataset showed similar black artifacts (or noise) in both two cases (SH2RGB and random RGB) as in the second video you showed.

Not fully solved. But for the Lego scene, initialize the RGB value of the points to 127 (narmalized: 0.5) can avoid such artifacts.