Using panoptic segmentation in Python

[manika1511]

I am trying to use Panoptic Deeplab in python, but I am facing the following error:

Loading config configs_panoptic/panoptic_deeplab_R_52_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.yaml with yaml.unsafe_load. Your machine may be at risk if the file contains malicious content.
The checkpoint state_dict contains keys that are not used by the model:
  pixel_mean
  pixel_std
Traceback (most recent call last):
  File "run_panoptic_segmentation.py", line 24, in <module>
    outputs = predictor("sample_images/ant+hill_10.jpg")
  File "/home/makapoor/anaconda3/envs/pytorch/lib/python3.6/site-packages/detectron2/engine/defaults.py", line 312, in __call__
    original_image = original_image[:, :, ::-1]
TypeError: string indices must be integers

from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.data import MetadataCatalog
import cv2
from detectron2.projects import panoptic_deeplab
import torch
import numpy as np

cfg = get_cfg()
panoptic_deeplab.add_panoptic_deeplab_config(cfg)
cfg.merge_from_file("panoptic_deeplab_R_52_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = "model_final_5e6da2.pkl"

predictor = DefaultPredictor(cfg)
outputs = predictor("image.jpg")

Also, I couldn't find any tutorial or documentation on how to use Panoptic-Deeplab in Python.

[bowenc0221]

Panoptic-DeepLab is based on Detectron2, you can refer to Detectron2 tutorial: https://detectron2.readthedocs.io/en/latest/

[manika1511]

@bowenc0221 can you suggest why I might be facing the above error? My input is an RGB image.

[bowenc0221]

Check this: https://github.com/facebookresearch/detectron2/blob/master/docs/tutorials/models.md#use-a-model

The input to DefaultPredictor is not filename.

[manika1511]

Thanks!!