NVIDIAGameWorks / kaolin-wisp

NVIDIA Kaolin Wisp is a PyTorch library powered by NVIDIA Kaolin Core to work with neural fields (including NeRFs, NGLOD, instant-ngp and VQAD).
Other
1.45k stars 133 forks source link

Initialize OctreeGrid with SPC Features #182

Open alexanderswerdlow opened 11 months ago

alexanderswerdlow commented 11 months ago

I'm trying to build a generalizable NeRF [e.g., pixelNeRF] where the features are not scene specific but taken from some encoder. I'm trying to use an SPC Field with an OctreeGrid to do this but it's unclear how one would go about this.

I wasn't able to find any model or pipeline that supports feedforward neural models. One hacky way I was trying to build a PoC was by modifying init_feature_structure() but I can't seem to properly initialize the structure given an existing SPC w/features.

Any help or pointers would be greatly appreciated!