If my image contains multiple people, how can I get the mask of a single person?

[womuer]

Now I can get a mulitple masks, can I get a single person mask? this is my code: for i, image_name in tqdm(enumerate(image_names), total=len(image_names)): print(image_name) image_path = os.path.join(input_dir, image_name) result = inference_model(model, image_path) print("predict picture") output_file = os.path.join(output_root, os.path.basename(image_path).replace('.jpg', '.png').replace('.jpeg', '.png').replace('.png', '.npy')) output_seg_file = os.path.join(output_root, os.path.basename(image_path).replace('.jpg', '.png').replace('.jpeg', '.png').replace('.png', '_seg.npy'))

    image = cv2.imread(image_path)

    pred_sem_seg = result.pred_sem_seg.data[0].cpu().numpy()  # H x W. seg ids.
    mask = (pred_sem_seg > 0)
    random_number = random.randint(1000, 9999)
    mask_image_path = os.path.join(output_root, f"{random_number}_mask.png")
    cv2.imwrite(mask_image_path, (mask * 255).astype(np.uint8))
    np.save(output_file, mask)
    np.save(output_seg_file, pred_sem_seg)

[Shaheerahmadzai]

Hi, I‌ tried to run the demo_seg_vis.py, this may run using this CLI‌ command: python3 demo/demo_seg_vis.py /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/configs/sapiens_seg/goliath/sapiens_0.3b_goliath-1024x768.py /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/sapiens_host/seg/checkpoints/sapiens_0.3b/sapiens_0.3b_goliath_best_goliath_mIoU_7673_epoch_194.pth --input /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/INPUT/img2.jpg --output_root /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/OUTPUT/img5.jpg

This is the result in the terminal: (sapiens-full) sahmadzai@nextcloud:~/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg$ python3 demo/demo_seg_vis.py /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/configs/sapiens_seg/goliath/sapiens_0.3b_goliath-1024x768.py /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/sapiens_host/seg/checkpoints/sapiens_0.3b/sapiens_0.3b_goliath_best_goliath_mIoU_7673_epoch_194.pth --input /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/INPUT/img2.jpg --output_root /home/sahmadzai/00_PROJECT4/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/OUTPUT/ /home/sahmadzai/miniconda3/envs/sapiens-full/lib/python3.8/site-packages/albumentations/init.py:13: UserWarning: A new version of Albumentations is available: 1.4.16 (you have 1.4.15). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1. check_for_updates() /home/sahmadzai/miniconda3/envs/sapiens-full/lib/python3.8/site-packages/torchvision/datapoints/init.py:12: UserWarning: The torchvision.datapoints and torchvision.transforms.v2 namespaces are still Beta. While we do not expect major breaking changes, some APIs may still change according to user feedback. Please submit any feedback you may have in this issue: https://github.com/pytorch/vision/issues/6753, and you can also check out https://github.com/pytorch/vision/issues/7319 to learn more about the APIs that we suspect might involve future changes. You can silence this warning by calling torchvision.disable_beta_transforms_warning(). warnings.warn(_BETA_TRANSFORMS_WARNING) /home/sahmadzai/miniconda3/envs/sapiens-full/lib/python3.8/site-packages/torchvision/transforms/v2/init.py:54: UserWarning: The torchvision.datapoints and torchvision.transforms.v2 namespaces are still Beta. While we do not expect major breaking changes, some APIs may still change according to user feedback. Please submit any feedback you may have in this issue: https://github.com/pytorch/vision/issues/6753, and you can also check out https://github.com/pytorch/vision/issues/7319 to learn more about the APIs that we suspect might involve future changes. You can silence this warning by calling torchvision.disable_beta_transforms_warning(). warnings.warn(_BETA_TRANSFORMS_WARNING) No module named 'airstore' Warning! Make sure you are not training. Loads checkpoint by local backend from path: /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/sapiens_host/seg/checkpoints/sapiens_0.3b/sapiens_0.3b_goliath_best_goliath_mIoU_7673_epoch_194.pth 0it [00:00, ?it/s]

It's just going to create an empty folder in the OUTPUT directory named img5.jpg can someone tell me, what is going on??

[Shaheerahmadzai]

And how to process a video instead of image??

[Shaheerahmadzai]

I‌ solved the above problem by running this CLI‌ command: python3 demo/demo_seg_vis.py /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/configs/sapiens_seg/goliath/sapiens_0.3b_goliath-1024x768.py /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/sapiens_host/seg/checkpoints/sapiens_0.3b/sapiens_0.3b_goliath_best_goliath_mIoU_7673_epoch_194.pth --input /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/INPUT/images.txt --output_root /home/sahmadzai/00_PROJECT/00_FULLBODY/SAPIENS_H_R_M_HUMAN_TASKS/sapiens/seg/OUTPUT/new

I create an images.txt file in the INPUT‌ directory and copy the image paths to this txt file right now it's processing the images and making masks

[rawalkhirodkar]

@Shaheerahmadzai our models are trained for semantic-segmentation and not instance segmentation. For this reason, it is non-trivial to get the single person mask. One way would be run an instance segmentor such as MaskRCNN and then compare with sapiens predictions.