MarcusAndreasSvensson / gaussian-splatting-webgpu

https://splatting.marcussvensson.com
MIT License
53 stars 5 forks source link

Black Stripes in Demo #1

Open JohannesKrueger opened 1 year ago

JohannesKrueger commented 1 year ago

Hello, i tried using the Demo (https://splatting.marcussvensson.com/) with the example data any custom data on a rtx 3070 but the results were always completely black or consited of black horizontal lines. Is there a solutions to this? Great Project! error_image

MarcusAndreasSvensson commented 1 year ago

Hmm, interesting... It does look like a variant to and issue i highlight in the readme that is basically caused by a large amount of intersections, which is absolutely solvable, but I didn't manage yet.

That issue can look something like this, and the pattern changes every frame I move.

image

Please try running the bonsai scene. I've never encountered any issues with that one. Do you see any warnings or errors in the log?

You could also try turning down the scale of the gaussians. If it gets solved by that we know that it's that same issue

JohannesKrueger commented 1 year ago

Thank you. I tried the Bonsai Scene. The Result ist better but still contains the black horizontal stripes. There are no errors in the log. I tried scaling down the gaussians but this didnt solved the issue. Is this the same error which you descriped as "partial blackness" in the "Caveats" ?. bonsai_error_image

MarcusAndreasSvensson commented 1 year ago

It doesn't look the same as the issue I described. You can see in the image I attached a clear view of where the screen tiles are located. In your case there's more clear lines though. With that said there is a clear improvement on the bonsai scene which i suspect has a lower data count which leads me to believe they might be the same issue, just looking a bit different.

I could try adding controls for custom image resolution prior to loading a scene and try lower it on your machine. The resolution is directly tied to the issue om referencing. I'll keep you posted.

JohannesKrueger commented 1 year ago

Thank You!

MarcusAndreasSvensson commented 1 year ago

Hello Johannes,

I opted to go for a fairly aggressive opacity culling instead to reduce the total number of intersections that has to be sorted. (Which is the reason for the partial blackness described)

Please try again and let me know if your problem persists.

MarcusAndreasSvensson commented 1 year ago

Unfortunately it seems like this problem is unrelated to what we have been speaking about.

I got access to a Windows machine today and it seems like some mismatch between Windows and Apple snuck in during my cleaning process.

I will look into it further.

MarcusAndreasSvensson commented 1 year ago

Hey @JohannesKrueger

I have now merged a new version of the rasterization shader which at least on my machine has resolved the issue.

Please let me know if it works fine for you as well.

JohannesKrueger commented 1 year ago

It works great with files with "larger size". I've tested multiple scenes. scenes with .ply file< 150mb causes black stripes in most cases, while scenes with .ply files >300mb always work perfectly.

MarcusAndreasSvensson commented 1 year ago

Alright, good to know we're on the right track!

Would you mind providing one or some of the failing files? I don't have any smaller files on hand

JohannesKrueger commented 1 year ago

Yes of course! I've uploaded one .ply file to this google drive link which causes the bug here is the Link: https://drive.google.com/file/d/1_FHSmsQfbO1NfHG5iqyXcGFALV5ypTbG/view?usp=sharing

MarcusAndreasSvensson commented 1 year ago

Thanks! I can reproduce the error which is great.

I'm working on a great performance improvement and by the looks of it it seems that it actually will solve this issue as well.

I'll keep you posted.