Question? Where should I look in the source code to see all the splats?

[lewibs]

I would like to play around with changing colors and removing splats from the Gaussian splat. Where would you recommend looking in the code for that?

[mkkellogg]

SplatMesh.js is the file that contains the core of the rendering code, specifically in the vertex shader: https://github.com/mkkellogg/GaussianSplats3D/blob/77b4af26ef138aeb05323f64b0eb5410d562f3a7/src/SplatMesh.js#L136 as well as the fragment shader: https://github.com/mkkellogg/GaussianSplats3D/blob/77b4af26ef138aeb05323f64b0eb5410d562f3a7/src/SplatMesh.js#L341

[mkkellogg]

I'll close this for now, but let me know if you need any more guidance with the code.

[lewibs]

finally got the chance to look into this. Been playing around with the shaders to change the color.

Some things I'm looking for is where is the Root truth for a color? is it in one of the buffers and what is the structure of those? And do you have any way you reccomend looking points that are around other splats? So if I raycast and get an index how possible would it be to get points around it? Would I need to implement kNN and just run it on all the points or does that already exist?

[lewibs]

@mkkellogg unsure if closed issues notify you so I'm just pinging

[lewibs]

Update: found that material.uniforms.centersColorsTextureValue is the way to go. Issue is however, when I update it, the scene does not seem to update as well. I've tested setting needs Update to true but no success.

[mkkellogg]

So for the "real" color of a splat, the source of truth is in the SplatBuffer object for a given splat scene. You can retrieve just color data from a splat buffer by calling the fillSplatColorArray() function.

For updating the color values, do you want to replace the whole texture or just update a sub-section of it? If you want to update/replace the whole texture, you need to do something like:

material.uniforms.centersColorsTexture.value = newtexture;
material.uniforms.centersColorsTexture.value.needsUpdate = true;

If you want to just update a sub-section of the texture, there are a couple ways to do it. You can use three.js' built-in method WebGLRenderer.copyTextureToTexture() (which I haven't personally used). Internally that function just calls the WebGL function texSubImage2D() to update a texture. I update the data textures in SplatMesh by calling texSubImage2D() directly in SplatMesh.updateDataTexture()

For finding splats near a given splat, I don't think there's currently a great built-in way to do it. However, since all splats are organized into an octree structure (SplatTree), it shouldn't be too hard to code up a search algorithm to find neighboring octree nodes for a given splat :)

[lewibs]

how would you recommend updating that data in splatBuffer? Im seeing in the code that there is lots more then color in the data. You get colors specifically by using fillSplatColorArray it seems. Would It be best to modify the splatBuffer, or update the textures that are stored in the mesh?

If i do update this.splatDataTextures.baseData.colors then do the actions to update the material, would that update the splats colors?

[lewibs]

@mkkellogg what im looking at right now is fillSplatColorArray. Im trying to figure out how this creates the uint4array but it seems to be dependent on the to and from input info. How would you reccomend getting a uint8array of all the colors in the mesh?

additionally, it looks like just calling refreshGPUDataFromSplatBuffers will update the mesh too. Is this the best way or is there a faster method?

[mkkellogg]

fillSplatColorArray() accepts as its first argument a parameter called outColorArray. That array should be an instance of Uint8Array, and it will get filled with color data from the SplatBuffer using the range values you specify (srcFrom & srcTo). destFrom is the start position in outColorArray to begin populating data.

Once the scene is completely loaded, this.splatDataTextures.baseData.colors will have all the color data extracted & uncompressed from the splat buffer. There is not currently a function that will just take that extracted color data and push it to the texture on the GPU, but you could totally write one :) It would be a lot like updateDataTextures(), except it wouldn't pull any data from the splat buffer. Actually updateDataTextures() should probably be renamed something like updateDataTexturesFromSplatBuffers() and it could call two new functions: one to extract & decompress the splat buffer data and another to push it to the GPU.

[lewibs]

I got the function working for changing the color thanks. One thing to note, im not sure the updateDataTextures() works.

when I change the data using: for all points... this.material.uniforms.centersColorsTexture.value.source.data.data[colorDestBase] = rgbaArrayToInteger([r,g,b,a], 0); this.material.uniforms.centersColorsTexture.value.needsUpdate = true;

then call mesh.udpateDataTextures() the colors dont revert back to their original color. Is that expected behavior? I would assume since I changed the colors away from the splatBuffer it would have gone back to the original color

[mkkellogg]

So you're right about updateDataTextures(), it works only during the progressive loading of a scene (I never intended it to be used after that). In theory, it should be able to be modified to work as I previously claimed it did. In many places you'll see references to this.lastBuildSplatCount and splatCount or splatCount - 1 as parameters to function calls within updateDataTextures() -- Those are just specifying a range of splats to do some update stuff with. When a scene is fully loaded, lastBuildSplatCount should be equal to splatCount so that's why you aren't seeing any effects. If you change those values (as appropriate) to be your desired range, that should make it work.

[lewibs]

I should be able to do a pr. just wanna make sure its implemented well. One thing im working on now is implementing that tree search. Im using visitLeaves but that seems to only include a small portion of the nodes. Is there a better function?

this was done using the SplatMesh.splatTree I assumed it would have been everything?

[lewibs]

my bad. looks like the node has node.data.indexes... no need to answer