search

sharinka0715 / semantic-gaussians

Official implemetation of the paper "Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting".
MIT License
143 stars 11 forks source link

Difference between rendering results and intermediate results #19

Closed HuGhcat-code closed 3 months ago

HuGhcat-code commented 3 months ago

Dear author: Thank you for open-source such a great work! When i was fusing the guitar scene you gave me(running fusion.py),I saved the intermediate semantic maps as shown below(one of many): image The other maps are basicly alike. After fusion,I ran view_viser.py,only to see a different semantic rendering result! image Some parts that are clearly not segmented in the intermediate semantic map are shown in the final result I want to know why Appreciate your help

sharinka0715 commented 3 months ago

Hi, I don't think it is possible that classes which are not segmented by VLPart can be shown in the final result. However, classes that are distinguished by VLPart is not necessarily to be in the intermediate segment map.

You should refer to the code of semantic map generation from VLPart. All parts segmented by VLPart could overlap, and the CLIP embeddings of these overlapped areas are averaged.

If you want to see the raw output of VLPart, please try to show the masks and instances from here and here.