search

Jumpat / SegmentAnythingin3D

Segment Anything in 3D with NeRFs (NeurIPS 2023)
Apache License 2.0
831 stars 52 forks source link

How to visualize the seg result of SA_GS? #68

Open Hendery77 opened 2 months ago

Hendery77 commented 2 months ago

Thanks for the great work! I am planing to reproduce your segment_result of SA_GS, however, It seems that you store the 3D mask as the seg_point_cloud.ply document, and your render.py use " pre_mask = torch.load(precomputed_mask)" to load the 3D mask ,it doesn't make senses, could you tell me how to reproduce your segment_result?

Thank you very much!

Jumpat commented 2 months ago

Hello, do you mean the SAGA? Or SA3D-GS? What do you want to get? The 2D rendered mask or the segmented target (with background removed)?

VolkaJ commented 2 months ago

Hello, do you mean the SAGA? Or SA3D-GS? What do you want to get? The 2D rendered mask or the segmented target (with background removed)?

I'd like to know SAGA version too. I think it's 3D Mask (3D Segementation possibly?)

Jumpat commented 2 months ago

The 3d mask is stored as confidence score in the seg_point_cloud.ply, each point in this point cloud has a mask attribute. In SAGA we use different realization, where we store another one mask file separately.

VolkaJ commented 2 months ago

The 3d mask is stored as confidence score in the seg_point_cloud.ply, each point in this point cloud has a mask attribute. In SAGA we use different realization, where we store another one mask file separately.

But in nerfstudio version, the only output is the mp4 file in train_vis folder right?

Jumpat commented 1 month ago

Yeah, but the 3D-GS version of our code is independent of nerfstudio. It is implemented based on the original 3D-GS