search

Jumpat / SegmentAnythingin3D

Segment Anything in 3D with NeRFs (NeurIPS 2023)
Apache License 2.0
869 stars 54 forks source link

Can it be Nerf agnostic? #2

Closed antoinemacia closed 1 year ago

antoinemacia commented 1 year ago

Hello hello

This looks amazing and very exciting, thank your for your work!

Question: Can it be agnostic of Nerf? Im particularly interested by the "3D mask grids via density-guided inverse rendering" technique, Im wondering if a set of 2D masks produced by SAM for a given object could output an assessment in 3D space regardless of Nerf (e.g a bounding box. or a serie of points)

Hope that makes sense, thanks!

Jumpat commented 1 year ago

Thanks for your interest! The mask inverse rendering is based on volume rendering and cannot output other data format. But we believe that if the 3D occupancy of an object is obtained via the mask inverse rendering (i.e., the mask grid), it would be convenient to get its corresponding bounding box or any other representation.😊

Jumpat commented 1 year ago

The supporting for other NeRFs will be realized in our future integration to NeRFStudio. Close this as completed! Feel free to ask any other question.