To get a segmentation label map from the model output

[rose-jinyang]

Hello How are you? Thanks for contributing to this project. Could u guide me to get a segmentation label map from the model output?

[praeclarumjj3]

Hi @rose-jinyang, thanks for your interest in our work.

You can run the demo script to visualize the segmentation label map from the model output. You may add an output_folder argument while initializing an evaluator object to specify the path where the segmentation label maps for the model's predictions should be saved.

https://github.com/SHI-Labs/OneFormer/blob/761189909f392a110a4ead574d85ed3a17fbc8a7/train_net.py#L358

becomes:

 evaluator = cls.build_evaluator(cfg, dataset_name, output_folder=<PATH-TO-YOUR-FOLDER>)

[rose-jinyang]

Thanks for your reply. I want to get the segmentation label map as numpy array on python code.

[praeclarumjj3]

Could you specify the segmentation task: semantic, instance, or panoptic?

[rose-jinyang]

I want to get instance segmentation label maps.

[rose-jinyang]

Or I want to get box, score, class_id and binary mask for each instance object.

[praeclarumjj3]

Thanks for the clarification. You can obtain tensors (convert into np.array) for those from the instance predictions directly.

instances = predictions["instances"]
boxes = instances.pred_boxes
scores = instances.scores
classes = instances.pred_classes
masks = instances.pred_masks

Note that to obtain boxes, you must set DETECTION_ON as True in the config. https://github.com/SHI-Labs/OneFormer/blob/761189909f392a110a4ead574d85ed3a17fbc8a7/configs/coco/oneformer_R50_bs16_50ep.yaml#L54

[rose-jinyang]

Thanks

[rose-jinyang]

Hello I have more questions. I checked the detectorn2's output format in their help. Here what is the N? Does the "pred_masks" mean a binary mask for each instance object in the rectangle regions by the "pred_boxes"?

[praeclarumjj3]

Hi @rose-jinyang, the N denotes the number of segmented regions (or object masks) in the image.

Does the "pred_masks" mean a binary mask for each instance object in the rectangle regions by the "pred_boxes"?

Yes, you are correct.

[praeclarumjj3]

Closing this issue. Feel free to re-open if you have any more questions.

[rose-jinyang]

Hi Thanks for your kind help. I want to get one segmentation map containing all the class labels rather than binary mask for each instance. How can I get it?