search

NVIDIAGameWorks / kaolin-wisp

NVIDIA Kaolin Wisp is a PyTorch library powered by NVIDIA Kaolin Core to work with neural fields (including NeRFs, NGLOD, instant-ngp and VQAD).
Other
1.45k stars 132 forks source link

Best way generate a video. #126

Open aakash94 opened 1 year ago

aakash94 commented 1 year ago

New to this library, can use some guidance here.

I have a dataset created using colman script. How to best train/load/save model so that I can query it to get images for different viewpoints, and then stitch it to get a video? Thanks!

orperel commented 1 year ago

Hi @aakash94 !

Training and eval should work as usual, treat this as a standard NeRF dataset (pass --multiview-dataset-format standard).

You can follow the NeRF Hash instructions within the NeRF app README:

Forward-facing scene, like the fox scene from Instant-NGP repository, are also supported.

Our code supports any "standard" NGP-format datasets that has been converted with the scripts from the instant-ngp library. We pass in the --multiview-dataset-format argument to specify the dataset type, which in this case is different from the RTMV dataset type used for the other examples.

The --mip argument controls the amount of downscaling that happens on the images when they get loaded. This is useful for datasets with very high resolution images to prevent overload on system memory, but is usually not necessary for reasonably sized images like the fox dataset.

If you're using the colmap script from instant-ngp, also take care to run it with the right aabb_scale (see their instructions on how to pick this one).