facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.55k stars 7.49k forks source link

riding_horse.png: detected 0 instances in 0.10s #4502

Open doantientai opened 2 years ago

doantientai commented 2 years ago

Hi everyone,

I followed the instructions in here for the installation. Things went well. But then I tried running the first demo. The model did not detect any object.

I have made no change to the code.

Instructions To Reproduce the Issue:

pyenv local 3.7.6
poetry init
poetry shell
python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
cd detectron2/demo/
python demo.py --config-file ../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml   --input  ~/Downloads/riding_horse.png

And this is the Full logs of what I got

[08/25 10:53:42 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='../configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['/home/tai/Downloads/riding_horse.png'], opts=[], output=None, video_input=None, webcam=False)
[08/25 10:53:44 fvcore.common.checkpoint]: [Checkpointer] Loading from detectron2://ImageNetPretrained/MSRA/R-50.pkl ...
[08/25 10:53:44 d2.checkpoint.c2_model_loading]: Renaming Caffe2 weights ......
[08/25 10:53:44 d2.checkpoint.c2_model_loading]: Following weights matched with submodule backbone.bottom_up:
| Names in Model    | Names in Checkpoint      | Shapes                                          |
|:------------------|:-------------------------|:------------------------------------------------|
| res2.0.conv1.*    | res2_0_branch2a_{bn_*,w} | (64,) (64,) (64,) (64,) (64,64,1,1)             |
| res2.0.conv2.*    | res2_0_branch2b_{bn_*,w} | (64,) (64,) (64,) (64,) (64,64,3,3)             |
| res2.0.conv3.*    | res2_0_branch2c_{bn_*,w} | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| res2.0.shortcut.* | res2_0_branch1_{bn_*,w}  | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| res2.1.conv1.*    | res2_1_branch2a_{bn_*,w} | (64,) (64,) (64,) (64,) (64,256,1,1)            |
| res2.1.conv2.*    | res2_1_branch2b_{bn_*,w} | (64,) (64,) (64,) (64,) (64,64,3,3)             |
| res2.1.conv3.*    | res2_1_branch2c_{bn_*,w} | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| res2.2.conv1.*    | res2_2_branch2a_{bn_*,w} | (64,) (64,) (64,) (64,) (64,256,1,1)            |
| res2.2.conv2.*    | res2_2_branch2b_{bn_*,w} | (64,) (64,) (64,) (64,) (64,64,3,3)             |
| res2.2.conv3.*    | res2_2_branch2c_{bn_*,w} | (256,) (256,) (256,) (256,) (256,64,1,1)        |
| res3.0.conv1.*    | res3_0_branch2a_{bn_*,w} | (128,) (128,) (128,) (128,) (128,256,1,1)       |
| res3.0.conv2.*    | res3_0_branch2b_{bn_*,w} | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| res3.0.conv3.*    | res3_0_branch2c_{bn_*,w} | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| res3.0.shortcut.* | res3_0_branch1_{bn_*,w}  | (512,) (512,) (512,) (512,) (512,256,1,1)       |
| res3.1.conv1.*    | res3_1_branch2a_{bn_*,w} | (128,) (128,) (128,) (128,) (128,512,1,1)       |
| res3.1.conv2.*    | res3_1_branch2b_{bn_*,w} | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| res3.1.conv3.*    | res3_1_branch2c_{bn_*,w} | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| res3.2.conv1.*    | res3_2_branch2a_{bn_*,w} | (128,) (128,) (128,) (128,) (128,512,1,1)       |
| res3.2.conv2.*    | res3_2_branch2b_{bn_*,w} | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| res3.2.conv3.*    | res3_2_branch2c_{bn_*,w} | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| res3.3.conv1.*    | res3_3_branch2a_{bn_*,w} | (128,) (128,) (128,) (128,) (128,512,1,1)       |
| res3.3.conv2.*    | res3_3_branch2b_{bn_*,w} | (128,) (128,) (128,) (128,) (128,128,3,3)       |
| res3.3.conv3.*    | res3_3_branch2c_{bn_*,w} | (512,) (512,) (512,) (512,) (512,128,1,1)       |
| res4.0.conv1.*    | res4_0_branch2a_{bn_*,w} | (256,) (256,) (256,) (256,) (256,512,1,1)       |
| res4.0.conv2.*    | res4_0_branch2b_{bn_*,w} | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| res4.0.conv3.*    | res4_0_branch2c_{bn_*,w} | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| res4.0.shortcut.* | res4_0_branch1_{bn_*,w}  | (1024,) (1024,) (1024,) (1024,) (1024,512,1,1)  |
| res4.1.conv1.*    | res4_1_branch2a_{bn_*,w} | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| res4.1.conv2.*    | res4_1_branch2b_{bn_*,w} | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| res4.1.conv3.*    | res4_1_branch2c_{bn_*,w} | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| res4.2.conv1.*    | res4_2_branch2a_{bn_*,w} | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| res4.2.conv2.*    | res4_2_branch2b_{bn_*,w} | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| res4.2.conv3.*    | res4_2_branch2c_{bn_*,w} | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| res4.3.conv1.*    | res4_3_branch2a_{bn_*,w} | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| res4.3.conv2.*    | res4_3_branch2b_{bn_*,w} | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| res4.3.conv3.*    | res4_3_branch2c_{bn_*,w} | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| res4.4.conv1.*    | res4_4_branch2a_{bn_*,w} | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| res4.4.conv2.*    | res4_4_branch2b_{bn_*,w} | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| res4.4.conv3.*    | res4_4_branch2c_{bn_*,w} | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| res4.5.conv1.*    | res4_5_branch2a_{bn_*,w} | (256,) (256,) (256,) (256,) (256,1024,1,1)      |
| res4.5.conv2.*    | res4_5_branch2b_{bn_*,w} | (256,) (256,) (256,) (256,) (256,256,3,3)       |
| res4.5.conv3.*    | res4_5_branch2c_{bn_*,w} | (1024,) (1024,) (1024,) (1024,) (1024,256,1,1)  |
| res5.0.conv1.*    | res5_0_branch2a_{bn_*,w} | (512,) (512,) (512,) (512,) (512,1024,1,1)      |
| res5.0.conv2.*    | res5_0_branch2b_{bn_*,w} | (512,) (512,) (512,) (512,) (512,512,3,3)       |
| res5.0.conv3.*    | res5_0_branch2c_{bn_*,w} | (2048,) (2048,) (2048,) (2048,) (2048,512,1,1)  |
| res5.0.shortcut.* | res5_0_branch1_{bn_*,w}  | (2048,) (2048,) (2048,) (2048,) (2048,1024,1,1) |
| res5.1.conv1.*    | res5_1_branch2a_{bn_*,w} | (512,) (512,) (512,) (512,) (512,2048,1,1)      |
| res5.1.conv2.*    | res5_1_branch2b_{bn_*,w} | (512,) (512,) (512,) (512,) (512,512,3,3)       |
| res5.1.conv3.*    | res5_1_branch2c_{bn_*,w} | (2048,) (2048,) (2048,) (2048,) (2048,512,1,1)  |
| res5.2.conv1.*    | res5_2_branch2a_{bn_*,w} | (512,) (512,) (512,) (512,) (512,2048,1,1)      |
| res5.2.conv2.*    | res5_2_branch2b_{bn_*,w} | (512,) (512,) (512,) (512,) (512,512,3,3)       |
| res5.2.conv3.*    | res5_2_branch2c_{bn_*,w} | (2048,) (2048,) (2048,) (2048,) (2048,512,1,1)  |
| stem.conv1.norm.* | res_conv1_bn_*           | (64,) (64,) (64,) (64,)                         |
| stem.conv1.weight | conv1_w                  | (64, 3, 7, 7)                                   |
WARNING [08/25 10:53:44 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint:
backbone.fpn_lateral2.{bias, weight}
backbone.fpn_lateral3.{bias, weight}
backbone.fpn_lateral4.{bias, weight}
backbone.fpn_lateral5.{bias, weight}
backbone.fpn_output2.{bias, weight}
backbone.fpn_output3.{bias, weight}
backbone.fpn_output4.{bias, weight}
backbone.fpn_output5.{bias, weight}
proposal_generator.rpn_head.anchor_deltas.{bias, weight}
proposal_generator.rpn_head.conv.{bias, weight}
proposal_generator.rpn_head.objectness_logits.{bias, weight}
roi_heads.box_head.fc1.{bias, weight}
roi_heads.box_head.fc2.{bias, weight}
roi_heads.box_predictor.bbox_pred.{bias, weight}
roi_heads.box_predictor.cls_score.{bias, weight}
roi_heads.mask_head.deconv.{bias, weight}
roi_heads.mask_head.mask_fcn1.{bias, weight}
roi_heads.mask_head.mask_fcn2.{bias, weight}
roi_heads.mask_head.mask_fcn3.{bias, weight}
roi_heads.mask_head.mask_fcn4.{bias, weight}
roi_heads.mask_head.predictor.{bias, weight}
WARNING [08/25 10:53:44 fvcore.common.checkpoint]: The checkpoint state_dict contains keys that are not used by the model:
  fc1000.{bias, weight}
  stem.conv1.bias
/media/tai/6TB/Projects/Segmentation/Detectron2/.venv/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
[08/25 10:53:44 detectron2]: /home/tai/Downloads/riding_horse.png: detected 0 instances in 0.10s

Expected behavior:

The model is expected to detect objects as being shown in the Colaboratory example with multiple detected objects.

Environment:

2022-08-25 11:03:19 URL:https://raw.githubusercontent.com/facebookresearch/detectron2/main/detectron2/utils/collect_env.py [8391/8391] -> "collect_env.py" [1]
----------------------  -----------------------------------------------------------------------------------------------------------
sys.platform            linux
Python                  3.7.6 (default, Nov  3 2021, 18:42:57) [GCC 9.3.0]
numpy                   1.21.6
detectron2              0.6 @/media/tai/6TB/Projects/Segmentation/Detectron2/detectron2/detectron2
Compiler                GCC 10.3
CUDA compiler           CUDA 10.2
detectron2 arch flags   6.1
DETECTRON2_ENV_MODULE   <not set>
PyTorch                 1.10.2+cu102 @/media/tai/6TB/Projects/Segmentation/Detectron2/.venv/lib/python3.7/site-packages/torch
PyTorch debug build     False
GPU available           Yes
GPU 0                   NVIDIA GeForce GTX 1080 Ti (arch=6.1)
Driver version          515.65.01
CUDA_HOME               /usr/local/cuda-10.2
Pillow                  9.2.0
torchvision             0.11.3+cu102 @/media/tai/6TB/Projects/Segmentation/Detectron2/.venv/lib/python3.7/site-packages/torchvision
torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5
fvcore                  0.1.5.post20220512
iopath                  0.1.9
cv2                     4.6.0
----------------------  -----------------------------------------------------------------------------------------------------------
PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

Hopefully someone know how to fix this. Thanks a lot for your attetion! :smiley:

github-actions[bot] commented 2 years ago

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template. The following information is missing: "Instructions To Reproduce the Issue and Full Logs";

doantientai commented 2 years ago

One thing I'm wondering is that, in the output of collect_env.py, we have

Compiler                GCC 10.3

But when I do gcc --version, it gives

gcc (Ubuntu 8.4.0-3ubuntu2) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

One more detail, I have multiple version of CUDA and GCC and I manage them using sudo update-alternatives --config cuda and sudo update-alternatives --config gcc. I don't know if this may be the cause?

kimile599 commented 2 years ago

Exactly same issue here: ckbone.fpn_lateral5. | backbone.fpn_lateral5.{bias,weight} | (256,) (256,2048,1,1) | | backbone.fpn_output2. | backbone.fpn_output2.{bias,weight} | (256,) (256,256,3,3) | | backbone.fpn_output3. | backbone.fpn_output3.{bias,weight} | (256,) (256,256,3,3) | | backbone.fpn_output4. | backbone.fpn_output4.{bias,weight} | (256,) (256,256,3,3) | | backbone.fpn_output5. | backbone.fpn_output5.{bias,weight} | (256,) (256,256,3,3) | | proposal_generator.rpn_head.anchor_deltas. | proposal_generator.rpn_head.anchor_deltas.{bias,weight} | (12,) (12,256,1,1) | | proposal_generator.rpn_head.conv. | proposal_generator.rpn_head.conv.{bias,weight} | (256,) (256,256,3,3) | | proposal_generator.rpn_head.objectness_logits. | proposal_generator.rpn_head.objectness_logits.{bias,weight} | (3,) (3,256,1,1) | | roi_heads.box_head.fc1. | roi_heads.box_head.fc1.{bias,weight} | (1024,) (1024,12544) | | roi_heads.box_head.fc2. | roi_heads.box_head.fc2.{bias,weight} | (1024,) (1024,1024) | | roi_heads.mask_head.deconv. | roi_heads.mask_head.deconv.{bias,weight} | (256,) (256,256,2,2) | | roi_heads.mask_head.mask_fcn1. | roi_heads.mask_head.mask_fcn1.{bias,weight} | (256,) (256,256,3,3) | | roi_heads.mask_head.mask_fcn2. | roi_heads.mask_head.mask_fcn2.{bias,weight} | (256,) (256,256,3,3) | | roi_heads.mask_head.mask_fcn3. | roi_heads.mask_head.mask_fcn3.{bias,weight} | (256,) (256,256,3,3) | | roi_heads.mask_head.mask_fcn4.* | roi_heads.mask_head.mask_fcn4.{bias,weight} | (256,) (256,256,3,3) | WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (4,) in the checkpoint but (320,) in the model! You might want to double check if this is expected. WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (4, 1024) in the checkpoint but (320, 1024) in the model! You might want to double check if this is expected. WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (2,) in the checkpoint but (81,) in the model! You might want to double check if this is expected. WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (2, 1024) in the checkpoint but (81, 1024) in the model! You might want to double check if this is expected. WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Skip loading parameter 'roi_heads.mask_head.predictor.bias' to the model due to incompatible shapes: (1,) in the checkpoint but (80,) in the model! You might want to double check if this is expected. WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Skip loading parameter 'roi_heads.mask_head.predictor.weight' to the model due to incompatible shapes: (1, 256, 1, 1) in the checkpoint but (80, 256, 1, 1) in the model! You might want to double check if this is expected. WARNING [08/29 07:34:11 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: roi_heads.box_predictor.bbox_pred.{bias, weight} roi_heads.box_predictor.cls_score.{bias, weight} roi_heads.mask_head.predictor.{bias, weight} 0% 0/1 [00:00<?, ?it/s]/usr/local/lib/python3.7/dist-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2894.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] [08/29 07:34:12 detectron2]: /content/dataset/test/DSC00280.JPG: detected 0 instances in 1.55s

I know that it is the config file doesn't match our trained model, " incompatible shapes: (1, 256, 1, 1) in the checkpoint but (80, 256, 1, 1) ", the 1 should be my one class target, but the 80 should be coco dataet with 80 classes, based on my knowledge. But i failed to solve it. I have tried added the : cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1 predictor = DefaultPredictor(cfg)

But no luck.

kimile599 commented 2 years ago

Update: I can use the my own trained weight to see the segmentation results using:

IMAGE_DIR='/content/dataset/test' file_names = os.listdir(IMAGE_DIR)

print(file_names)

for file_names in os.listdir(IMAGE_DIR):

print(d["file_name"])

#print(os.path.basename(d["file_name"]))    
im = cv2.imread(os.path.join(IMAGE_DIR, file_names))
outputs = predictor(im)  # format is documented at https://detectron2.readthedocs.io/tutorials/models.html#model-output-format
v = Visualizer(im[:, :, ::-1],
               metadata=my_val_metadata, 
               scale=0.5, 
               instance_mode=ColorMode.IMAGE
               #instance_mode=ColorMode.IMAGE_BW
               # ColorMode.IMAGE_BW remove the colors of unsegmented pixels. This option is only available for segmentation models
)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2_imshow(out.get_image()[:, :, ::-1])

all instances were correct, but when i test the same image with python demo.py, I got 0 instances. Just want to figure it out where i might get wrong

EnnaSachdeva commented 2 years ago

I'm also facing the same issue. Any update on this?

maidouxiaozi commented 2 years ago

I'm also facing the same issue. Any update on this?

slent310 commented 3 months ago

I'm facing the same issue. [0m[Checkpointer] Loading from C:\Users\xxx/.torch/iopath_cache\detectron2/ImageNetPretrained/MSRA\R-50.pkl ... [08/02 11:12:53 d2.checkpoint.c2_model_loading]: Renaming Caffe2 weights ...... [08/02 11:12:53 d2.checkpoint.c2_model_loading]: Following weights matched with submodule backbone.bottom_up - Total num: 54 WARNING [08/02 11:12:58 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint: In my case, I find that the checkpoint seem not loading. so I put model pkl file to C:/Users/.torch/iopath_cache/detectron2/ImageNetPretrained/MSRA/ and name it R-50.pkl then it works. hope it helps.