search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.6k stars 7.5k forks source link

RuntimeError: Currently topk on mps works only for k<=16 in Apple silicon GPU (device = MPS) #4342

Open bikcrum opened 2 years ago

bikcrum commented 2 years ago

Instructions To Reproduce the 🐛 Bug:

I am trying to run detectron2 demo code from google colab in my system which has apple silicon GPU but I am encountering an error.

Reproduction steps:

1. Download sample image from coco dataset using the following command:

!wget http://images.cocodataset.org/val2017/000000439715.jpg -q -O input.jpg

2. Run the following code:

It is the same code from the front page. Link: https://github.com/facebookresearch/detectron2#:~:text=%2C%20and%20the-,Colab%20Notebook,-to%20learn%20about

# Some basic setup:
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger

setup_logger()

# import some common libraries
import numpy as np
import os, json, cv2, random
import matplotlib.pyplot as plt

# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog

cfg = get_cfg()

# This line is added to set device to "mps"
cfg.MODEL.DEVICE = "mps"

# add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model
# Find a model from detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
predictor = DefaultPredictor(cfg)

# !wget http://images.cocodataset.org/val2017/000000439715.jpg -q -O input.jpg
im = cv2.imread("input.jpg")
plt.imshow(im)

outputs = predictor(I'm)
print(outputs)

3. Actual output

WARNING:root:Pytorch pre-release version 1.13.0.dev20220618 - assuming intent to test it
/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:2890.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "/Users/bikram/PycharmProjects/detectron-example/main.py", line 33, in <module>
    outputs = predictor(im)
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/engine/defaults.py", line 317, in __call__
    predictions = self.model([inputs])[0]
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1131, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward
    return self.inference(batched_inputs)
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 208, in inference
    proposals, _ = self.proposal_generator(images, features, None)
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1131, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn.py", line 478, in forward
    anchors, pred_objectness_logits, pred_anchor_deltas, images.image_sizes
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn.py", line 511, in predict_proposals
    self.training,
  File "/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/proposal_utils.py", line 79, in find_top_rpn_proposals
    topk_scores_i, topk_idx = logits_i.topk(num_proposals_i, dim=1)
RuntimeError: Currently topk on mps works only for k<=16

Expected output:

When the device is set to "CPU" on the same system, I get the following outputs without any error:

{'instances': Instances(num_instances=15, image_height=480, image_width=640, fields=[pred_boxes: Boxes(tensor([[126.6035, 244.8977, 459.8291, 480.0000],
         [251.1083, 157.8127, 338.9731, 413.6379],
         [114.8496, 268.6864, 148.2352, 398.8111],
         [  0.8217, 281.0327,  78.6072, 478.4210],
         [ 49.3954, 274.1229,  80.1545, 342.9808],
         [561.2248, 271.5816, 596.2755, 385.2552],
         [385.9072, 270.3125, 413.7130, 304.0397],
         [515.9295, 278.3744, 562.2792, 389.3803],
         [335.2409, 251.9167, 414.7491, 275.9375],
         [350.9300, 269.2060, 386.0984, 297.9081],
         [331.6292, 230.9996, 393.2759, 257.2009],
         [510.7349, 263.2656, 570.9865, 295.9194],
         [409.0841, 271.8646, 460.5582, 356.8722],
         [506.8767, 283.3257, 529.9403, 324.0392],
         [594.5663, 283.4820, 609.0577, 311.4124]])), scores: tensor([0.9997, 0.9957, 0.9915, 0.9882, 0.9861, 0.9840, 0.9769, 0.9716, 0.9062,
         0.9037, 0.8870, 0.8575, 0.6592, 0.5899, 0.5767]), pred_classes: tensor([17,  0,  0,  0,  0,  0,  0,  0, 25,  0, 25, 25,  0,  0, 24]), pred_masks: tensor([[[False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          ...,
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False]],

         [[False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          ...,
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False]],

         [[False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          ...,
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False]],

         ...,

         [[False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          ...,
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False]],

         [[False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          ...,
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False]],

         [[False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          ...,
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False],
          [False, False, False,  ..., False, False, False]]])])}

Environment:

Provide your environment information using the following command:

---------------------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sys.platform           darwin
Python                 3.7.13 (default, Mar 28 2022, 07:24:34) [Clang 12.0.0 ]
numpy                  1.21.6
detectron2             0.6 @/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2
detectron2._C          not built correctly: dlopen(/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/_C.cpython-37m-darwin.so, 0x0002): Symbol not found: __ZN2at5emptyEN3c108ArrayRefIxEENS0_13TensorOptionsENS0_8optionalINS0_12MemoryFormatEEE
                         Referenced from: /Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/detectron2/_C.cpython-37m-darwin.so
                         Expected in: /Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/torch/lib/libtorch_cpu.dylib
Compiler ($CXX)        Apple clang version 13.1.6 (clang-1316.0.21.2.5)
DETECTRON2_ENV_MODULE  <not set>
PyTorch                1.13.0.dev20220618 @/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/torch
PyTorch debug build    False
GPU available          No: torch.cuda.is_available() == False
Pillow                 9.1.1
torchvision            0.14.0.dev20220618 @/Users/bikram/miniforge3/envs/mcs2/lib/python3.7/site-packages/torchvision
fvcore                 0.1.5.post20220512
iopath                 0.1.9
cv2                    4.6.0
---------------------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
PyTorch built with:
  - GCC 4.2
  - C++ Version: 201402
  - clang 13.1.6
  - Intel(R) Math Kernel Library Version 2020.0.1 Product Build 20200208 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201811
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/Applications/Xcode_13.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -Wno-deprecated-declarations -DUSE_PTHREADPOOL -Xpreprocessor -fopenmp -I/usr/local/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_PYTORCH_METAL_EXPORT -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -DUSE_COREML_DELEGATE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-range-loop-analysis -Wno-pass-failed -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -DUSE_MPS -fno-objc-arc -Wno-unused-private-field -Wno-missing-braces -Wno-c++14-extensions -Wno-constexpr-not-const, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
bikcrum commented 2 years ago

Related: https://github.com/pytorch/pytorch/issues/78915

albirrkarim commented 1 year ago

any solution for this?

SlothFriend commented 1 year ago

I'm also running into this

kulinseth commented 1 year ago

@bikcrum can you try the latest nightly or the pytorch 2.0 release . This issue should be resolved

agilests commented 1 year ago

env: mac m1 pytorch 2.2.0.dev20231123 python 3.7 The problem still exists