aras-p / UnityGaussianSplatting

Toy Gaussian Splatting visualization in Unity
MIT License
2.09k stars 239 forks source link

PLY import: support non-standard data ordering (e.g. scaniverse) #108

Open benoitmar opened 6 months ago

benoitmar commented 6 months ago

Hello,

I get this while importing ply from scaniverse. PLY Vertex size mismatch, expected 248 but file has 236.

Can you help? Thanks

Ben

aras-p commented 6 months ago

That sounds exactly like https://github.com/aras-p/UnityGaussianSplatting/issues/81#issuecomment-2033537936 i.e. the PLY file is "somehow" not matching the standard data layout, but hard to say how exactly without having the file in question. Can you share some file from that scaniverse thing?

benoitmar commented 6 months ago

Thanks for you fast answer. Here is a sample file

Regards Ben

aras-p commented 6 months ago

Ok looks like scaniverse produces a PLY file with different data order compared to what most other 3DGS software does. To fix this, I'd have to implement some code to rearrange the data into the expected order. I don't know if/when I'd do that (or someone could implement this and contribute a PR to this repo?). This could also perhaps be fixed on scaniverse side by making their export match the data order used by others.

Scaniverse data layout is this:

float3 pos;
float3 scale;
float4 rot;
float opacity;
float3 color;
float3 sh[15];

whereas the "standard" order is:

float3 pos;
float3 normal;
float3 color;
float3 sh[15];
float opacity;
float3 scale;
float4 rot;
jackjansen commented 5 months ago

I'm having the same issue.

I've sent an email to Scaniverse, asking what they can tell me about their format. I'll report back here if I get an answer.

I thought of fixing it myself (or maybe writing a ply-file converter), but I can't find the normal data in the scaniverse files.

Incidentally, that's also the reason the Scaniverse files give an error: each point is 3 floats short of the expected size (hence 12 bytes shorter).

aras-p commented 5 months ago

@jackjansen oh the scaniverse format/layout is known (see my previous comment here). The normal in the "standard" 3DGS PLY layout is not used for anything whatsoever, just putting zeroes in there would work fine.

benoitmar commented 5 months ago

I have reported the format issue to the Scanniverse team, it should be fixed in the next release.

mrintergalactickeyboard commented 5 months ago

I attempted to patch GaussSplatAssetCreator.cs and GaussSplatRendererEditor.cs to load the original Scaniverse file format, but just disabling the normals check and changing the data format didn't work. The color and transparency did not load correctly. I was able to fix problematic Scaniverse ply files and load in Unity after converting them from 3DGS format to Cloud Compare and back to 3DGS using 3dgsconverter. HTH anyone waiting on the Scaniverse app update.

keithito commented 5 months ago

Hi! The latest version of Scaniverse (3.0.2) exports to PLY with the "standard" layout and is compatible with UnityGaussianSplatting.