search

Harry-Zhi / semantic_nerf

The implementation of "In-Place Scene Labelling and Understanding with Implicit Scene Representation" [ICCV 2021].
Other
418 stars 54 forks source link

Questions about NDC space #19

Closed Maguire1999 closed 1 year ago

Maguire1999 commented 2 years ago

Hi, thanks for the great work! It seems that semantic-nerf doesn't seem to use NDC space, I want to know why?

Harry-Zhi commented 2 years ago

Hi @Maguire1999 ,

The NDC space used in NeRF is mainly designed for training MLP in LLFF-type unbounded scenes where all cemeras face towards certain directions (you can image that you have a plane spliiting the whole 3D world into two parts, one side with content you are interested, one side with cameras facing towards the content/objects). As as result, you could warp the distant world space into a bounded volume through parametrazation.

In Semantic-NeRF, we assume a more general in-/out-ward looking scenerio where camera floats within the scene with arbitrary poses. In this case the NDC is not suited anymore (you could not find such a plane spliiting the world into two parts with contents and cameras only, respectively). In this case, I use the basic form where the samples spread within a near and far thresholds.

Hope it helps.

Maguire1999 commented 2 years ago

It helps me a lot! Thank you!