Unable to load ROI

[TemitopeOladokun]

Hello @TannerGilbert I am presently working with detectron but I got the below message during the training. I am using "COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml" for the checkpoint. What am I doing wrong?

model_final_a3ec72.pkl: 254MB [00:18, 13.4MB/s]
Unable to load 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 1024) in the checkpoint but (18, 1024) in the model! Unable to load 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (81,) in the checkpoint but (18,) in the model! Unable to load 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 1024) in the checkpoint but (68, 1024) in the model! Unable to load 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (320,) in the checkpoint but (68,) in the model! Unable to load 'roi_heads.mask_head.predictor.weight' to the model due to incompatible shapes: (80, 256, 1, 1) in the checkpoint but (17, 256, 1, 1) in the model! Unable to load 'roi_heads.mask_head.predictor.bias' to the model due to incompatible shapes: (80,) in the checkpoint but (17,) in the model!

[TannerGilbert]

You need to change the NUM_CLASSES before training.

Example:

cfg.MODEL.ROI_HEADS.NUM_CLASSES = <num_classes>

For more information please check out the Training the model section of my Detectron2 Train a Instance Segmentation Model article.

[TemitopeOladokun]

Thank you @TannerGilbert for your reply. I changed the number of classes to the number of my classes but still continue to get this error.

[TannerGilbert]

Can you send me your code

[TemitopeOladokun]

`from detectron2.engine import DefaultTrainer from detectron2.config import get_cfg import os

cfg = get_cfg() cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml")) cfg.DATASETS.TRAIN = ("my_data_train",) cfg.DATASETS.TEST = () # no metrics implemented for this dataset cfg.DATALOADER.NUM_WORKERS = 2 cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml") # initialize from model zoo cfg.SOLVER.IMS_PER_BATCH = 2 cfg.SOLVER.BASE_LR = 0.005 cfg.SOLVER.MAX_ITER = 50000 # 300 iterations seems good enough, but you can certainly train longer cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 256 # faster, and good enough for this toy dataset cfg.MODEL.ROI_HEADS.NUM_CLASSES = 18 # 18 number of classes

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True) trainer = DefaultTrainer(cfg) trainer.resume_or_load(resume=False) trainer.train()`

[TannerGilbert]

Can you also send me the code where you load in your data-set.

[TannerGilbert]

Eventhough it doesn't help with the error you should probably remove cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 256 because this doesn't work well with larger datasets.

[TemitopeOladokun]

`from detectron2.data.datasets import register_coco_instances register_coco_instances("my_data_train", {}, "/content/drive/My Drive/image/train/train.json", "/content/drive/My Drive/image/train")

my_data_train_metadata = MetadataCatalog.get("my_data_train") dataset_dicts = DatasetCatalog.get("my_data_train")`

[TannerGilbert]

Hard to say where the problem lies. Does your data look correct if you visualize it?

[TemitopeOladokun]

Do you mean after the conversion to COCO? @TannerGilbert I used Tony607 script for conversion. I could register the instance too.

[TannerGilbert]

I meant if you can run

import random
from detectron2.data import DatasetCatalog, MetadataCatalog

dataset_dicts = DatasetCatalog.get("my_data_train")
microcontroller_metadata = MetadataCatalog.get("my_data_train")

for d in random.sample(dataset_dicts, 3):
    img = cv2.imread(d["file_name"])
    v = Visualizer(img[:, :, ::-1], metadata=microcontroller_metadata, scale=0.5)
    v = v.draw_dataset_dict(d)
    plt.figure(figsize = (14, 10))
    plt.imshow(cv2.cvtColor(v.get_image()[:, :, ::-1], cv2.COLOR_BGR2RGB))
    plt.show()

after you registered your data-set with

from detectron2.data.datasets import register_coco_instances
register_coco_instances("my_data_train", {}, "/content/drive/My Drive/image/train/train.json", "/content/drive/My Drive/image/train")

You can see how I did it in the following notebook: https://github.com/TannerGilbert/Detectron2-Train-a-Instance-Segmentation-Model/blob/master/Microcontroller_Instance_Segmentation_with_COCO_dataformat.ipynb

[TemitopeOladokun]

It works perfectly well.

[TannerGilbert]

Is training starting despite of the warning messages? Because I'm also getting this warning messages when I'm training the model and it doesn't seem to effect the training process.

[TannerGilbert]

Actually it seems like this warning is expected and doesn't have any bad effects. It only shows up because your dataset has different number of classes from the pre-trained model.

For more information check out: https://github.com/facebookresearch/detectron2/issues/196#issuecomment-547681921

[TemitopeOladokun]

I did not get any error but the "Unable to load ROI" which bothers me because it is not so in yours.

[TannerGilbert]

Actually I just think they changed the warning messages in the newer version, because I just tried it out and now my warnings look just the same as yours. Therefore you can just ignore them.

I will close this issue now. If you have any further questions feel free to ask.

[haneessh]

Hello @TannerGilbert and @TemitopeOladokun I have the same error, as expected there are no issues in training process, but when I use it on "how to predict frames from a video or a webcam stream", it is throwing error "Unable to load 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (3, 1024) in the checkpoint but (81, 1024) in the model! Unable to load 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (3,) in the checkpoint but (81,) in the model! Unable to load 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (8, 1024) in the checkpoint but (320, 1024) in the model! Unable to load 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (8,) in the checkpoint but (320,) in the model! Error opening video stream or file"

I used the same code which you had mentioned on "https://github.com/TannerGilbert/Microcontroller-Detection-with-Detectron2" "https://github.com/TannerGilbert/Microcontroller-Detection-with-Detectron2/blob/master/detect_from_webcam_or_video.py"

please help me on this.. I want to access webcam and detect.. Thanks in advance

[TemitopeOladokun]

I have not solved that error too @haneeshb.

In fact, I am about to ask if I should be bothered if my APs reads "NAN" @TannerGilbert