search

Pointcept / SegmentAnything3D

[ICCV'23 Workshop] SAM3D: Segment Anything in 3D Scenes
https://arxiv.org/abs/2306.03908
MIT License
949 stars 67 forks source link

How to get SAM Generate Masks #50

Open Yoow4 opened 2 months ago

Yoow4 commented 2 months ago

Hi, I would like to know more details about how to use SAM to get the segmentation masks on 2D frames and then map them into the 3D space via depth information. When I use it, SAM sets the following parameters mask_generator = SamAutomaticMaskGenerator( model=sam, points_per_side=64, pred_iou_thresh=0.86, box_nms_thresh= 0.7, stability_score_thresh=0.92, crop_n_layers=1, crop_n_points_downscale_factor=2, min_mask_region_area=200, # Requires open-cv to run post-processing ) But I can't get the same segmentation effect as you, how can I set the parameters and use the depth map to project to 3D space。 Hope to get your answer. Thanks.

yhyang-myron commented 2 months ago

Hi, we get 3D point clouds with 2D RGB, depth and camera parameters at here. And we process 2D masks at here.

Yoow4 commented 1 month ago

Hi, For the last step, when I run sam3d.py I get the.pth file. I look at the pcd_ensemble function with open(org_path) as f: segments = json.load(f) It needs a.json file.

What should I do to get the.json file.

flybiubiu commented 1 month ago

Hi, For the last step, when I run sam3d.py I get the.pth file. I look at the pcd_ensemble function with open(org_path) as f: segments = json.load(f) It needs a.json file.

What should I do to get the.json file.

+1