search

Harry-Zhi / semantic_nerf

The implementation of "In-Place Scene Labelling and Understanding with Implicit Scene Representation" [ICCV 2021].
Other
426 stars 56 forks source link

Questions about Multi-view Semantic Fusion #11

Closed Unrealluver closed 2 years ago

Unrealluver commented 2 years ago

Hi, I am your sincere follower! Your semantic-NeRF is a very suprising work！

I wonder if you could release the Multi-view Semantic Fusion code?

Harry-Zhi commented 2 years ago

Hi @Unrealluver,

Thanks for your insterests and sorry for my late reply as I way away office for last few months.

Do you mean the multi-view Bayesian or Average fusion implementation? Given depth and poses, you can establish the dense correspondences between views and simply fuse multi-view softmax probablities using either multiplication (Bayesian fusion) or aveage before normalisation.

Some related implementation could be found here: https://github.com/lvzhaoyang/DeeperInverseCompositionalAlgorithm/blob/9a401bf2b03ecfed169c1a655b4fe8be8a4c211d/code/models/geometry.py, where you can use the (inverse) warping function to merge multiview predcitions given depth and camera poses of views.

Hope it helps!