mosure / bevy_gaussian_splatting

bevy gaussian splatting render pipeline plugin
https://mosure.github.io/bevy_gaussian_splatting?gaussian_count=1000
MIT License
130 stars 9 forks source link

Initialising Cloud with rotated GlobalTransform messes up rendering #108

Closed teodosin closed 2 months ago

teodosin commented 2 months ago

This is something I noticed just now after trying to load in my own gcloud files. They're generated with Polycam, and their Z axis is inverted from the one that this plugin uses. My quick method of rotating the cloud was with the following function:

fn setup_gaussian(
    mut commands: Commands,
    asset_server: Res<AssetServer>,
){
    commands.spawn((
        GaussianSplattingBundle {
            cloud: asset_server.load("Trii.gcloud"),
            settings: GaussianCloudSettings {

                global_transform: GlobalTransform::from(Mat4 {
                    x_axis: Vec4::new(1.0, 0.0, 0.0, 0.0),
                    y_axis: Vec4::new(0.0, -1.0, 0.0, 0.0),
                    z_axis: Vec4::new(0.0, 0.0, -1.0, 0.0),
                    w_axis: Vec4::new(0.0, 0.0, 0.0, 1.0),
                }),
                ..Default::default()
            },
            ..Default::default()
        },
    ));
    commands.spawn((
        Camera3dBundle::default(),
        PanOrbitCamera::default(),
    ));
}

Initialising the GlobalTransform of the cloud with a 4x4 matrix. The rotation is successful, but the splats get culled at odd angles where they're definitely supposed to be visible. It seems like the rendering is not respecting the GlobalTransform component on the cloud and assumes it's always default.

Before image

After. Same angle, but the trees are getting cut off from their roots or trunks and a bunch of other gaussians are getting culled unnecessarily. image

For context, the same angle in Polycam's web viewer. image

This angle makes it seem like the issue is with the sorting algorithm. The trees are getting cut off where the hill behind them begins. image

A related but separate issue: should GaussianClouds use the Transform component like built-in Bevy primitives?