search

nv-tlabs / XCube

[CVPR 2024 Highlight] XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies
https://research.nvidia.com/labs/toronto-ai/xcube/
Other
344 stars 20 forks source link

How to provide the user-guided edited voxel grid as input #9

Closed tanghaotommy closed 4 months ago

tanghaotommy commented 4 months ago

Dear authors,

I found the idea of user-guided editing very interesting and would like to try the example showed in Figure 6. I can build the voxel grid using Goxel, but the input to the diffusion model is a grid with normal as features. How do I get the normal from the voxel grid? I am new to this field, sorry if the answer is obvious. Thank you!

xrenaa commented 4 months ago

Hi, for that experiments, we trained a model without normal as condition. For easier editing, you could try to train a sparse VAE from 16 -> 128 and then train a 16x16x16 sparse Diffusion model. Then you could play with this sparse diffusion model and change the condition to it by Goxel.

tanghaotommy commented 4 months ago

Thanks a lot for the information! I'll give it a try and reach back if I may have other questions.