[for help] RTMDetIns visualization isu

[Wooho-Moon]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] I have read the FAQ documentation but cannot get the expected help.
• [X] The bug has not been fixed in the latest version (master) or latest version (3.x).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

master branch https://github.com/open-mmlab/mmdetection

Environment

sys.platform: linux Python: 3.7.16 (default, Jan 17 2023, 22:20:44) [GCC 11.2.0] CUDA available: True numpy_random_seed: 2147483648 GPU 0,1,2,3: NVIDIA GeForce RTX 3090 CUDA_HOME: /usr/local/cuda NVCC: Cuda compilation tools, release 11.3, V11.3.109 GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.11.0 PyTorch compiling details: PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.3
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.2
• Magma 2.5.2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.12.0 OpenCV: 4.7.0 MMEngine: 0.6.0 MMDetection: 3.0.0rc6+61dd8d5

Reproduces the problem - code sample

import torch, torchvision import mmdet import mmcv import mmengine

import mmcv import mmengine

from mmdet.apis import init_detector, inference_detector from mmdet.utils import register_all_modules

Choose to use a config and initialize the detector

config_file = '/home/workspace/segmantation/instance/mmdetection/configs/rtmdet/rtmdet-ins_x_8xb16-300e_coco.py'

Setup a checkpoint file to load

checkpoint_file = '/home/workspace/segmantation/instance/mmdeploy/checkpoint/rtmdet-ins_x_8xb16-300e_coco_20221124_111313-33d4595b.pth'

register all modules in mmdet into the registries

register_all_modules()

build the model from a config file and a checkpoint file

model = init_detector(config_file, checkpoint_file, device='cuda:0') # or device='cuda:0'

Use the detector to do inference

image = mmcv.imread('/home/workspace/segmantation/instance/mmdetection/demo/demo.jpg',channel_order='rgb') result = inference_detector(model, image) print(result)

from mmdet.registry import VISUALIZERS

init visualizer(run the block only once in jupyter notebook)

visualizer = VISUALIZERS.build(model.cfg.visualizer)

the dataset_meta is loaded from the checkpoint and

then pass to the model in init_detector

visualizer.dataset_meta = model.dataset_meta

show the results

visualizer.add_datasample( 'result', image, data_sample=result, draw_gt = None, wait_time=0, out_file=None, pred_score_thr=0.4 )

visualizer.show()

Reproduces the problem - command or script

it is not a command. it is run on jupyternotebook.(ipynb)

Reproduces the problem - error message

there is no error but the result is so wired.

the result is here

what should i do? how can I solve this problem?

Additional information

No response

[Wooho-Moon]

ah I use a model which is pretrained on this repository. the model is RTMDetIns-x, and I use a config file named rtmdet-ins_x_8xb16-300e_coco.py

[RangiLyu]

I copy your code and run it, but the result is correct.

import torch, torchvision
import mmdet
import mmcv
import mmengine

import mmcv
import mmengine

from mmdet.apis import init_detector, inference_detector
from mmdet.utils import register_all_modules

config_file = 'configs/rtmdet/rtmdet-ins_x_8xb16-300e_coco.py'
checkpoint_file = '~/models/rtmins/gather_model/rtmdet/rtmdet-ins_x_8xb16-300e_coco/rtmdet-ins_x_8xb16-300e_coco_20221124_111313-33d4595b.pth'
register_all_modules()
model = init_detector(config_file, checkpoint_file, device='cuda:0')  # or device='cuda:0'

image = mmcv.imread('demo/demo.jpg', channel_order='rgb')
result = inference_detector(model, image)
print(result)
from mmdet.registry import VISUALIZERS

visualizer = VISUALIZERS.build(model.cfg.visualizer)

visualizer.dataset_meta = model.dataset_meta
visualizer.add_datasample(
    'result',
    image,
    data_sample=result,
    draw_gt=None,
    wait_time=0,
    out_file=None,
    pred_score_thr=0.4
)

visualizer.show()

[Wooho-Moon]

umm.. ok.. I will try again. thanks alot:)

[ZhuYihan-UMI]

Hi Wooho-Moon, I met similar issue with you, and also carefully examine the codes, environments, changed a few computers to try, but could not solve the problem, and I searched all over the internet but finds no answers.

If you are also carefully examine the codes line by line, just like what I did, but still see errors, you may want to try the following solution.

Eventually the solutions turns out surprisingly simple: Just run all the scripts together. like what [RangiLyu] did. Use "run all" from the get-go.

Hope this helps.