search

Jumpat / SegmentAnythingin3D

Segment Anything in 3D with NeRFs (NeurIPS 2023)
Apache License 2.0
831 stars 52 forks source link

Difference between sa3d-gs and SAGA? #76

Open VolkaJ opened 3 weeks ago

VolkaJ commented 3 weeks ago

I guess SA3D and SAGA are two different methods to segment 3D objects. SA3D uses pre-trained Nerf & 3dgs model and SAGA uses 3dgs? Am I correct?

Jumpat commented 3 weeks ago

They are two different projects. You can refer to the papers of SA3D and SAGA for details.

VolkaJ commented 3 weeks ago

Thanks for your answer. I'm using the nerfstudio-version branch to use text prompt but I wanna improve processing time by replacing Nerf with 3dgs. Can you provide me a brief guide to do that?

Jumpat commented 3 weeks ago

This is not an easy issue. The nerf studio branch is maintained by my collaborator. The 3D-GS version currently does not support text prompts.

The text-prompt based segmentation is implemented by replacing the point prompts in the first frame provided by the user with Grounding-DINO output bounding box. A relatively easy way to implement it is on the SA3D-GS branch.

Currently, the SA3D-GS branch may have some bugs. We will fix them recently.