Sharp edges (non Gaussian)

antimatter15 / splat

WebGL 3D Gaussian Splat Viewer

https://antimatter15.com/splat/

MIT License

1.89k stars 193 forks source link

Sharp edges (non Gaussian) #2

Closed vincent-lecrubier-skydio closed 8 months ago

vincent-lecrubier-skydio commented 1 year ago

In many case, I can see sharp edges, here are a couple questions:

Is that expected, or a bug on my machine?
Is it due to taking some shortcut to favor performance (e.g. not doing gaussian sorting, but more traditional rasterization with z buffer)?
Do you have ideas on how to fix this?

Screenshot 2023-09-11 at 6 43 03 PM Screenshot 2023-09-11 at 6 44 31 PM Screenshot 2023-09-11 at 6 47 06 PM

antimatter15 commented 1 year ago

Right now it creates fragments that are roughly the same shape as the underlying gaussians, but especially for very squished gaussians the bounding box that it generates is too small. That the bounding box is small is partially because I am not sure about the math, and also partly because it speeds things up. This comment might be related: https://news.ycombinator.com/item?id=37473228

It is properly sorting the gaussians on a separate CPU thread (albeit at a lower frame rate than the actual render), so this isn't a z-buffer artifact.

JayFoxRox commented 1 year ago

It is properly sorting the gaussians on a separate CPU thread (albeit at a lower frame rate than the actual render), so this isn't a z-buffer artifact.

It would be good if there was some indication that sorting is still ongoing.

Firepal commented 1 year ago

You can increase the size of the quads to something bigger by changing the two 7.0 * in the vertex shader to something bigger.

andresrommier commented 1 year ago

You can increase the size of the quads to something bigger by changing the two 7.0 * in the vertex shader to something bigger.

Yes, this helps now the quality of the splats looks closer to what the original code and https://gsplat.tech/ but there is a big performance hit. I wonder if we can make this size dependent on the properties of the gaussians (not all gaussians need a very large area).

Firepal commented 1 year ago

I wonder if we can make this size dependent on the properties of the gaussians (not all gaussians need a very large area).

The coverage calculation math is already there, but it seems to be really, really iffy. It feels wrong. I think we'll have to borrow some code from elsewhere.

Firepal commented 1 year ago

On a totally-unrelated note, gsplat is amazing and should be open-source