search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.55k stars 7.49k forks source link

How test detectron2 models on sub classes COCO DATASET #4936

Closed bouachalazhar closed 1 year ago

bouachalazhar commented 1 year ago

I would like to test on this 17 classes, it's a custom dataset from COCO.

Instructions To Reproduce the Issue:

Check https://stackoverflow.com/help/minimal-reproducible-example for how to ask good questions. Simplify the steps to reproduce the issue using suggestions from the above link, and provide them below:

  1. Full runnable code or full changes you made: 
    
cfg = get_cfg()
try:
cfg.MODEL.DEVICE = DEVICE
except:
cfg.MODEL.DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
model_path = "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
cfg.merge_from_file(model_zoo.get_config_file(model_path))
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(model_path)
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model

thing_classes = ['person', 'bicycle', 'car', 'motorcycle', 'bench', 'horse', 'frisbee', 'skis', 'snowboard', 'sports ball', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'cup'] thing_id = [id for id, lab in enumerate(MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).thing_classes) if lab in thing_classes] thing_colors = [MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).thing_colors[id] for id in thing_id] thing_dataset_id_to_contiguous_id = {k:v for k,v in MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).thing_dataset_id_to_contiguous_id.items() if v in thing_id} metadata=metadata = MetadataCatalog.get(cfg.DATASETS.TEST).set(thing_classes=thing_classes, thing_colors=thing_colors, thing_dataset_id_to_contiguous_id=thing_dataset_id_to_contiguous_id) cfg.MODEL.ROI_HEADS.NUM_CLASSES = len(thing_classes) cfg.MODEL.NUM_CLASSES = len(thing_classes)

predictor = DefaultPredictor(cfg)


3. __Full logs__ or other relevant observations:

[05/04 00:06:44 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl ... Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 1024) in the checkpoint but (17, 1024) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (81,) in the checkpoint but (17,) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 1024) in the checkpoint but (64, 1024) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (320,) in the checkpoint but (64,) in the model! You might want to double check if this is expected. Some model parameters or buffers are not found in the checkpoint: roi_heads.box_predictor.bbox_pred.{bias, weight} roi_heads.box_predictor.cls_score.{bias, weight}


## Environment:

Paste the output of the following command:

No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1'

sys.platform win32 Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] numpy 1.23.4 detectron2 0.6 @C:\Users\Lazhar Bouacha\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\detectron2 Compiler MSVC 193532215 CUDA compiler not available DETECTRON2_ENV_MODULE PyTorch 1.13.1+cu117 @C:\Users\Lazhar Bouacha\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torch PyTorch debug build False torch._C._GLIBCXX_USE_CXX11_ABI False GPU available No: torch.cuda.is_available() == False Pillow 9.4.0 torchvision 0.14.1+cu117 @C:\Users\Lazhar Bouacha\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torchvision fvcore 0.1.5.post20221221 iopath 0.1.9 cv2 4.7.0

PyTorch built with:

If your issue looks like an installation issue / environment issue, please first check common issues in https://detectron2.readthedocs.io/tutorials/install.html#common-installation-issues

bouachalazhar commented 1 year ago

You can try it : 

def custom_metadata(metadata, classes, outputs):
    id_classes = {c:metadata.thing_classes.index(c) for c in classes if c in metadata.thing_classes}
    mask = list(outputs.keys())[0]
    indices = torch.where(outputs[mask].get_fields()['pred_classes'] == list(id_classes.values())[0])[0]
    for k in outputs[mask].get_fields().keys():
        if type(outputs[mask].get_fields()[k]) == torch.Tensor:
            outputs[mask].get_fields()[k] = outputs[mask].get_fields()[k][indices]
        else:
            outputs[mask].get_fields()[k].tensor = outputs[mask].get_fields()[k].tensor[indices]
    return outputs, mask

and use it : 

outputs = predictor(im)
outputs, mask = custom_metadata(metadata, classes, outputs)
v = Visualizer(im[:, :, ::-1], metadata=metadata, scale=1, instance_mode=ColorMode.IMAGE)
out = v.draw_instance_predictions(outputs[mask].to("cpu"))