mkkellogg / GaussianSplats3D

Three.js-based implementation of 3D Gaussian splatting
MIT License
1.37k stars 168 forks source link

Can not render this simplest GS. #202

Closed yuedajiong closed 5 months ago

yuedajiong commented 5 months ago

GS_make.zip

def gauss(color_rgb2sh=1): def sigmoid(x): return 1 / (1 + np.exp(-x)) def sigmoid_inverse(y): return np.log(y / (1 - y))

p = np.array([
        [0.0, 0.0, 0.0],
        [1.0, 0.0, 0.0],
        [0.0, 1.0, 0.0],
        [0.0, 0.0, 1.0],
    ])  #WANT: [-∞,+∞]            #REAL: [-1,+1]*N  by tanh()*N
c = np.array([
        [0.6, 0.5, 0.4], 
        [1.0, 0.0, 0.0], 
        [0.0, 1.0, 0.0], 
        [0.0, 0.0, 1.0],
    ])  #WANT: [0,1] by sigmoid()
a = np.array([
        [1.0],
        [0.9],
        [0.8],
        [0.7],
    ])  #WANT: [0,1] by sigmoid()
s = np.array([
        [0.030, 0.030, 0.030],
        [0.300, 0.300, 0.300],
        [0.001, 0.010, 0.100],
        [0.003, 0.030, 0.300],
    ])  #WANT: [-∞,+∞] by exp()    #REAL: [0.01,0.3]
r = np.array([
        [1.0, 0.0, 0.0, 0.0],
        [1.0, 0.0, 0.0, 0.0],
        [1.0, 0.0, 0.0, 0.0],
        [1.0, 0.0, 0.0, 0.0],
    ])  #WANT: [-1, 1]  #unit-quaternion  #torch.nn.functional.normalize

color_activation = ''  #sigmoid
if color_activation == 'sigmoid':
    c = sigmoid_inverse(c)
else:
    pass
if color_rgb2sh:
    c = (np.array(c)-0.5)/0.28209479177387814
else:
    pass

alpha_activation = ''  #sigmoid
if alpha_activation == 'sigmoid':
    a = sigmoid_inverse(a)
else:
    pass

scale_activation = 'exp'  #sigmoid
if scale_activation == 'sigmoid':
    s = sigmoid_inverse(s)
    scale_sigmoid_min = 0.001
    scale_sigmoid_max = 0.300
    s = scale_sigmoid_min + (scale_sigmoid_max-scale_sigmoid_min) * s
elif scale_activation == 'exp':
    s = np.log(s)
elif scale_activation == 'softplus':
    pass #TODO
else:
    s = s

return np.concatenate([p, c, a, s, r], axis=-1).astype(np.float32)
mkkellogg commented 5 months ago

I'm not sure what this code is supposed to do. Also, it's all python and my repo only contains Javascript... was this question meant for a different project?

yuedajiong commented 5 months ago

This python code generated a mini GS .ply. gs.zip

But different render/viewer generated different output image. Your viewer rendered: image

But we expected: image

mkkellogg commented 5 months ago

It looks like your .ply file is ASCII format, can you re-export in binary format (little endian) and try again? Currently my viewer only supports binary format.

yuedajiong commented 5 months ago

00 GS_make_view.zip

Thanks!!!

mkkellogg commented 5 months ago

All the .ply files in that .zip bundle are still ascii format :)

yuedajiong commented 5 months ago

Sorry, updated... :-)

00 GS_make_view.zip

mkkellogg commented 5 months ago

For some reason that just links to the github issue :)

yuedajiong commented 5 months ago

A zip, downloadable ...

mkkellogg commented 5 months ago

With that binary .ply I was able to see something similar to what you think you should be able to see: image The background is black because the default background for my viewer is black. (I don't currently have a way to change, that, I should probably add an option to set that).

yuedajiong commented 5 months ago

@mkkellogg

yes, GREAT-MASTER:

I found that the binary .ply is right, and the text .ply can not be supported. :-(

you can close this issue.

Thanks a lot.

mkkellogg commented 5 months ago

Binary PLY format makes more sense for large scenes since it requires much less space :)