Ruyi-Zha / r2_gaussian

Other
34 stars 2 forks source link

Gaussian Rasterizer Outputs only zeros values #3

Open vinith2 opened 9 hours ago

vinith2 commented 9 hours ago

I tried to use Guassian rasterizer on a set of random Gaussian values as below:

import torch 
import numpy as np
import matplotlib.pyplot as plt
DEVICE = 0
from xray_gaussian_rasterization_voxelization import (
    GaussianRasterizationSettings,
    GaussianRasterizer,
)

NUM_GUASSIAN = 10000

N1 = 256
N2 = 256
N3 = 256
# Generate random guassian centers
means = np.random.randn(NUM_GUASSIAN, 3) #-0.5

means[:,0] = means[:,0]*N1
means[:,1] = means[:,1]*N2
means[:,2] = means[:,2]*N3

means = torch.tensor(means, device=DEVICE,dtype=torch.float32,requires_grad=True)
opacities = np.random.rand(2,NUM_GUASSIAN)
opacities[:] = 100
opacities = torch.tensor(opacities, device=DEVICE,dtype=torch.float32, requires_grad=True)

covs = np.eye(3).repeat(NUM_GUASSIAN,1).reshape(NUM_GUASSIAN,3,3)*10

covs = torch.tensor(covs, device=DEVICE,dtype=torch.float32, requires_grad=True)

means2D = torch.zeros_like(means, device=DEVICE,dtype=torch.float32, requires_grad=True)

viewmat = torch.eye(4, device = DEVICE)
viewmat[0,3]=0
viewmat[1,3]=0
viewmat[2,3]=0

campos = torch.tensor([0.0, 0, 0]).to (DEVICE)  # Camera position

p = torch.eye(4).to(DEVICE)

raster_settings = GaussianRasterizationSettings(
        image_height=int(256),
        image_width=int(256),
        mode = 0,
        debug= 2,
        tanfovx = 1,
        tanfovy = 1,
        scale_modifier=1,
        prefiltered= False,
        campos= campos,
        projmatrix = p,
        viewmatrix= viewmat,
    )

rasterizer = GaussianRasterizer(raster_settings=raster_settings)

color,radii = rasterizer(
        means3D=means,
        means2D=means2D,
        opacities=opacities,
        scales=None,
        rotations=None,
        cov3D_precomp=covs,
    )

I get all zeros for the output variable color I tried varying the opacity and viewmats without any change in the output. The code doesn't throw an error or warning message as well.

Ruyi-Zha commented 3 hours ago

Hi, the issue might be related to camera parameters, such as projmax, campos, and viewmatrix.

I just updated the repo. Our initialize_pcd.py supports generating random point clouds for initialization. I suggest you generate random Gaussians with python initialize_pcd.py --data <data path> --output <data path/init_random.npy> --recon_method random --random_density_max 1.0.

Then you can train with python train.py -s <data path> --ply_path <data path/init_random.npy>.

Below are the rendered X-ray projections at the 1st iteration.

image