search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.49k stars 7.48k forks source link

Unable to use custom model (Some model parameters or buffers are not found in the checkpoint) #4172

Closed kimlia545 closed 2 years ago

kimlia545 commented 2 years ago

I want to use custom model trained on the GPU in local CPU

  1. Full runnable code or full changes you made: 
    
#TRAIN CODE
from detectron2.engine import DefaultTrainer

cfg = get_cfg() cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")) cfg.DATASETS.TRAIN = ("train",) cfg.DATASETS.TEST = ("valid",) cfg.DATALOADER.NUM_WORKERS = 2 cfg.MODEL.WEIGHTS = "/content/drive/MyDrive/detectron2/model_final_f10217.pkl" #model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml") # Let training initialize from model zoo cfg.SOLVER.IMS_PER_BATCH = 2 cfg.SOLVER.BASE_LR = 0.00025 # pick a good LR cfg.SOLVER.MAX_ITER = 300 # 300 iterations seems good enough for this toy dataset; you will need to train longer for a practical dataset cfg.SOLVER.STEPS = [] # do not decay learning rate cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128 # faster, and good enough for this toy dataset (default: 512) cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1 # only has one class (ballon). (see https://detectron2.readthedocs.io/tutorials/datasets.html#update-the-config-for-new-datasets) ###############################

NOTE: this config means the number of classes, but a few popular unofficial tutorials incorrect uses num_classes+1 here.

cfg.INPUT.MIN_SIZE_TRAIN = 480 # 960

cfg.INPUT.MAX_SIZE_TRAIN = 480 # 960

from detectron2.engine import DefaultTrainer from detectron2.evaluation import COCOEvaluator

class CocoTrainer(DefaultTrainer):

@classmethod def build_evaluator(cls, cfg, dataset_name, output_folder=None):

if output_folder is None:
    os.makedirs("coco_eval", exist_ok=True)
    output_folder = "coco_eval"

return COCOEvaluator(dataset_name, cfg, False, output_folder)

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True) os.environ['CUDA_LAUNCH_BLOCKING'] = "0" trainer = DefaultTrainer(cfg) trainer.resume_or_load(resume=False) trainer.train()

2. What exact command you run:

cfg = get_cfg()

cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth") cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.7 cfg.MODEL.DEVICE = "cpu"

cfg.MODEL.RPN.BBOX_REG_LOSS_TYPE = "smooth_l1"

cfg.MODEL.RPN.BBOX_REG_LOSS_WEIGHT = 1.0

cfg.MODEL.ROI_BOX_HEAD.BBOX_REG_LOSS_TYPE = "smooth_l1"

cfg.MODEL.ROI_BOX_HEAD.BBOX_REG_LOSS_WEIGHT = 1.0

cfg.DATASETS.TEST = ("valid", )

cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1

predictor = DefaultPredictor(cfg)

from detectron2.utils.visualizer import ColorMode

dataset_dicts = DatasetCatalog.get("valid") for d in random.sample(dataset_dicts, 3):
im = cv2.imread(d["file_name"]) outputs = predictor(im) print(outputs) v = Visualizer(im[:, :, ::-1], metadata=valid_metadata, scale=0.8, instance_mode=ColorMode.IMAGE_BW # remove the colors of unsegmented pixels ) v = v.draw_instance_predictions(outputs["instances"].to("cpu")) # “pred_boxes”, “pred_classes”, “scores”, “pred_masks” (or “pred_masks_rle”) cv2_imshow(v.get_image()[:, :, ::-1])


4. __Full logs__ or other relevant observations:

Skip loading parameter 'proposal_generator.rpn_head.conv.weight' to the model due to incompatible shapes: (256, 256, 3, 3) in the checkpoint but (1024, 1024, 3, 3) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.conv.bias' to the model due to incompatible shapes: (256,) in the checkpoint but (1024,) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.objectness_logits.weight' to the model due to incompatible shapes: (3, 256, 1, 1) in the checkpoint but (15, 1024, 1, 1) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.objectness_logits.bias' to the model due to incompatible shapes: (3,) in the checkpoint but (15,) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.anchor_deltas.weight' to the model due to incompatible shapes: (12, 256, 1, 1) in the checkpoint but (60, 1024, 1, 1) in the model! You might want to double check if this is expected. Skip loading parameter 'proposal_generator.rpn_head.anchor_deltas.bias' to the model due to incompatible shapes: (12,) in the checkpoint but (60,) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (2, 1024) in the checkpoint but (81, 2048) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (2,) in the checkpoint but (81,) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (4, 1024) in the checkpoint but (320, 2048) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (4,) in the checkpoint but (320,) in the model! You might want to double check if this is expected. Some model parameters or buffers are not found in the checkpoint: backbone.res2.0.conv1.norm.{bias, weight} backbone.res2.0.conv1.weight backbone.res2.0.conv2.norm.{bias, weight} backbone.res2.0.conv2.weight backbone.res2.0.conv3.norm.{bias, weight} backbone.res2.0.conv3.weight backbone.res2.0.shortcut.norm.{bias, weight} backbone.res2.0.shortcut.weight backbone.res2.1.conv1.norm.{bias, weight} backbone.res2.1.conv1.weight backbone.res2.1.conv2.norm.{bias, weight} backbone.res2.1.conv2.weight backbone.res2.1.conv3.norm.{bias, weight} backbone.res2.1.conv3.weight backbone.res2.2.conv1.norm.{bias, weight} backbone.res2.2.conv1.weight backbone.res2.2.conv2.norm.{bias, weight} backbone.res2.2.conv2.weight backbone.res2.2.conv3.norm.{bias, weight} backbone.res2.2.conv3.weight backbone.res3.0.conv1.norm.{bias, weight} backbone.res3.0.conv1.weight backbone.res3.0.conv2.norm.{bias, weight} backbone.res3.0.conv2.weight backbone.res3.0.conv3.norm.{bias, weight} backbone.res3.0.conv3.weight backbone.res3.0.shortcut.norm.{bias, weight} backbone.res3.0.shortcut.weight backbone.res3.1.conv1.norm.{bias, weight} backbone.res3.1.conv1.weight backbone.res3.1.conv2.norm.{bias, weight} backbone.res3.1.conv2.weight backbone.res3.1.conv3.norm.{bias, weight} backbone.res3.1.conv3.weight backbone.res3.2.conv1.norm.{bias, weight} backbone.res3.2.conv1.weight backbone.res3.2.conv2.norm.{bias, weight} backbone.res3.2.conv2.weight backbone.res3.2.conv3.norm.{bias, weight} backbone.res3.2.conv3.weight backbone.res3.3.conv1.norm.{bias, weight} backbone.res3.3.conv1.weight backbone.res3.3.conv2.norm.{bias, weight} backbone.res3.3.conv2.weight backbone.res3.3.conv3.norm.{bias, weight} backbone.res3.3.conv3.weight backbone.res4.0.conv1.norm.{bias, weight} backbone.res4.0.conv1.weight backbone.res4.0.conv2.norm.{bias, weight} backbone.res4.0.conv2.weight backbone.res4.0.conv3.norm.{bias, weight} backbone.res4.0.conv3.weight backbone.res4.0.shortcut.norm.{bias, weight} backbone.res4.0.shortcut.weight backbone.res4.1.conv1.norm.{bias, weight} backbone.res4.1.conv1.weight backbone.res4.1.conv2.norm.{bias, weight} backbone.res4.1.conv2.weight backbone.res4.1.conv3.norm.{bias, weight} backbone.res4.1.conv3.weight backbone.res4.2.conv1.norm.{bias, weight} backbone.res4.2.conv1.weight backbone.res4.2.conv2.norm.{bias, weight} backbone.res4.2.conv2.weight backbone.res4.2.conv3.norm.{bias, weight} backbone.res4.2.conv3.weight backbone.res4.3.conv1.norm.{bias, weight} backbone.res4.3.conv1.weight backbone.res4.3.conv2.norm.{bias, weight} backbone.res4.3.conv2.weight backbone.res4.3.conv3.norm.{bias, weight} backbone.res4.3.conv3.weight backbone.res4.4.conv1.norm.{bias, weight} backbone.res4.4.conv1.weight backbone.res4.4.conv2.norm.{bias, weight} backbone.res4.4.conv2.weight backbone.res4.4.conv3.norm.{bias, weight} backbone.res4.4.conv3.weight backbone.res4.5.conv1.norm.{bias, weight} backbone.res4.5.conv1.weight backbone.res4.5.conv2.norm.{bias, weight} backbone.res4.5.conv2.weight backbone.res4.5.conv3.norm.{bias, weight} backbone.res4.5.conv3.weight backbone.stem.conv1.norm.{bias, weight} backbone.stem.conv1.weight proposal_generator.rpn_head.anchor_deltas.{bias, weight} proposal_generator.rpn_head.conv.{bias, weight} proposal_generator.rpn_head.objectness_logits.{bias, weight} roi_heads.box_predictor.bbox_pred.{bias, weight} roi_heads.box_predictor.cls_score.{bias, weight} roi_heads.res5.0.conv1.norm.{bias, weight} roi_heads.res5.0.conv1.weight roi_heads.res5.0.conv2.norm.{bias, weight} roi_heads.res5.0.conv2.weight roi_heads.res5.0.conv3.norm.{bias, weight} roi_heads.res5.0.conv3.weight roi_heads.res5.0.shortcut.norm.{bias, weight} roi_heads.res5.0.shortcut.weight roi_heads.res5.1.conv1.norm.{bias, weight} roi_heads.res5.1.conv1.weight roi_heads.res5.1.conv2.norm.{bias, weight} roi_heads.res5.1.conv2.weight roi_heads.res5.1.conv3.norm.{bias, weight} roi_heads.res5.1.conv3.weight roi_heads.res5.2.conv1.norm.{bias, weight} roi_heads.res5.2.conv1.weight roi_heads.res5.2.conv2.norm.{bias, weight} roi_heads.res5.2.conv2.weight roi_heads.res5.2.conv3.norm.{bias, weight} roi_heads.res5.2.conv3.weight The checkpoint state_dict contains keys that are not used by the model: backbone.fpn_lateral2.{bias, weight} backbone.fpn_output2.{bias, weight} backbone.fpn_lateral3.{bias, weight} backbone.fpn_output3.{bias, weight} backbone.fpn_lateral4.{bias, weight} backbone.fpn_output4.{bias, weight} backbone.fpn_lateral5.{bias, weight} backbone.fpn_output5.{bias, weight} backbone.bottom_up.stem.conv1.weight backbone.bottom_up.stem.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.0.shortcut.weight backbone.bottom_up.res2.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.0.conv1.weight backbone.bottom_up.res2.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.0.conv2.weight backbone.bottom_up.res2.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.0.conv3.weight backbone.bottom_up.res2.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.1.conv1.weight backbone.bottom_up.res2.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.1.conv2.weight backbone.bottom_up.res2.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.1.conv3.weight backbone.bottom_up.res2.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.2.conv1.weight backbone.bottom_up.res2.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.2.conv2.weight backbone.bottom_up.res2.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res2.2.conv3.weight backbone.bottom_up.res2.2.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.0.shortcut.weight backbone.bottom_up.res3.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.0.conv1.weight backbone.bottom_up.res3.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.0.conv2.weight backbone.bottom_up.res3.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.0.conv3.weight backbone.bottom_up.res3.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.1.conv1.weight backbone.bottom_up.res3.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.1.conv2.weight backbone.bottom_up.res3.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.1.conv3.weight backbone.bottom_up.res3.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.2.conv1.weight backbone.bottom_up.res3.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.2.conv2.weight backbone.bottom_up.res3.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.2.conv3.weight backbone.bottom_up.res3.2.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.3.conv1.weight backbone.bottom_up.res3.3.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.3.conv2.weight backbone.bottom_up.res3.3.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res3.3.conv3.weight backbone.bottom_up.res3.3.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.0.shortcut.weight backbone.bottom_up.res4.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.0.conv1.weight backbone.bottom_up.res4.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.0.conv2.weight backbone.bottom_up.res4.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.0.conv3.weight backbone.bottom_up.res4.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.1.conv1.weight backbone.bottom_up.res4.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.1.conv2.weight backbone.bottom_up.res4.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.1.conv3.weight backbone.bottom_up.res4.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.2.conv1.weight backbone.bottom_up.res4.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.2.conv2.weight backbone.bottom_up.res4.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.2.conv3.weight backbone.bottom_up.res4.2.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.3.conv1.weight backbone.bottom_up.res4.3.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.3.conv2.weight backbone.bottom_up.res4.3.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.3.conv3.weight backbone.bottom_up.res4.3.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.4.conv1.weight backbone.bottom_up.res4.4.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.4.conv2.weight backbone.bottom_up.res4.4.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.4.conv3.weight backbone.bottom_up.res4.4.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.5.conv1.weight backbone.bottom_up.res4.5.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.5.conv2.weight backbone.bottom_up.res4.5.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res4.5.conv3.weight backbone.bottom_up.res4.5.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.0.shortcut.weight backbone.bottom_up.res5.0.shortcut.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.0.conv1.weight backbone.bottom_up.res5.0.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.0.conv2.weight backbone.bottom_up.res5.0.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.0.conv3.weight backbone.bottom_up.res5.0.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.1.conv1.weight backbone.bottom_up.res5.1.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.1.conv2.weight backbone.bottom_up.res5.1.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.1.conv3.weight backbone.bottom_up.res5.1.conv3.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.2.conv1.weight backbone.bottom_up.res5.2.conv1.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.2.conv2.weight backbone.bottom_up.res5.2.conv2.norm.{bias, running_mean, running_var, weight} backbone.bottom_up.res5.2.conv3.weight backbone.bottom_up.res5.2.conv3.norm.{bias, running_mean, running_var, weight} roi_heads.box_head.fc1.{bias, weight} roi_heads.box_head.fc2.{bias, weight} roi_heads.mask_head.mask_fcn1.{bias, weight} roi_heads.mask_head.mask_fcn2.{bias, weight} roi_heads.mask_head.mask_fcn3.{bias, weight} roi_heads.mask_head.mask_fcn4.{bias, weight} roi_heads.mask_head.deconv.{bias, weight} roi_heads.mask_head.predictor.{bias, weight}



## Expected behavior:
I trained a model with gpu in Google Colab. I want to use it on my local cpu.
When executed on the CPU, an empty result is returned.
{'instances': Instances(num_instances=0, image_height=1920, image_width=1080, fields=[pred_boxes: Boxes(tensor([], size=(0, 4))), scores: tensor([]), pred_classes: tensor([], dtype=torch.int64)])}
ppwwyyxx commented 2 years ago

The config values used in your training and inference code are different: your training config loads from COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml plus many other changes.

They need to match so the model architecture in training and inference are the same.

kimlia545 commented 2 years ago

Thank you! I solved the problem

# Save custom config
with open("mycfg.yaml", "w") as f: 
    f.write(cfg.dump())
# Load custom config
cfg = get_cfg()
cfg.merge_from_file("mycfg.yaml")
dipesh-commits commented 2 years ago

@kimlia545 I also got same issue. Should the custom config be save while training?