search

nv-tlabs / LION

Latent Point Diffusion Models for 3D Shape Generation
Other
740 stars 58 forks source link

Visualizing Latent points before decoding? #76

Open Javiergonb opened 1 month ago

Javiergonb commented 1 month ago

Hi again @ZENGXH,

Is there a way to visualize the z_local tensor that is created when running the DDPM on the encoded h0? When using the sample method of the LION model the output dict returns a z_local with (num_samples, 8192,1,1) dimensions. From my understanding this is the z_local after decoding? I am not sure how visualize in 3D the z_local that is created before the decoding, like how the show here:

image

How did you do it?

Thanks in advance!

ZENGXH commented 4 weeks ago

could you try to reshape it to z_reshape in (num_samples, 2048, 4,1,1) and then visualize z_reshape[:,:,:3] (the last channel is feature not xyz, so we dont plot it)? Let me know if it does not work

Javiergonb commented 4 weeks ago

Yes this works. I was trying to do something similar with the VAE. I am getting h0 from the recont method in the VAE model. This part of the code:

https://github.com/nv-tlabs/LION/blob/333c5b4903098418a2aae897ff079bcf0adc3eae/models/vae_adain.py#L202C16-L202C99

I believe this is getting a 3d representation of h0 no?