search

NVlabs / instant-ngp

Instant neural graphics primitives: lightning fast NeRF and more
https://nvlabs.github.io/instant-ngp
Other
16.04k stars 1.93k forks source link

Hi there, first of call: cool setup! #949

Open 0xTechSavvy opened 2 years ago

0xTechSavvy commented 2 years ago

Hi there, first of call: cool setup!

(1) shouldn't be a problem as long as the camera poses are precise enough for the Train extrinsics option to fix any slight inaccuracies.

I suspect the problem comes more from (2) and (3).

For (2), we recently (yesterday) pushed support for per-camera metadata. You can customize it via the python bindings

testbed.nerf.training.dataset.metadata[image_id].camera_distortion = ...
testbed.nerf.training.dataset.metadata[image_id].focal_length = ...
testbed.nerf.training.dataset.metadata[image_id].principal_point = ...

More specifics are in python_api.cu. The .json-based loader unfortunately only supports a single set of camera parameters per json file.

For (3), this will largely manifest as artifacts when trying to view the scene from the top of bottom (i.e. outside of the convex hull of the training data). If you plan for the viewpoint to stay close to the ring of cameras, you should be fine.

Curious to hear whether you find the Python bindings w.r.t. (2) helpful.

Cheers!

Originally posted by @Tom94 in https://github.com/NVlabs/instant-ngp/issues/257#issuecomment-1048803260

0xTechSavvy commented 2 years ago

Does it support per-camera metadata now?