search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.56k stars 7.49k forks source link

DensePose_CSE KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>" #4525

Open sword-shadow opened 2 years ago

sword-shadow commented 2 years ago

DensePose_CSE KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

  1. create a virtual environment and install the required libraries 
    conda create -n densepose python=3.7
conda activate densepose
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
pip install git+https://github.com/facebookresearch/detectron2@main#subdirectory=projects/DensePose
  2. try to inference an image with a pre-trained human CSE model, following the tutorial of https://github.com/facebookresearch/detectron2/blob/main/projects/DensePose/doc/GETTING_STARTED.md 
    cd detectron2/projects/DensePose
python apply_net.py show configs/cse/densepose_rcnn_R_50_FPN_DL_soft_s1x.yaml model_final_9199f5.pkl image.png dp_contour,bbox --output image_densepose_contour.png

    3. KeyError was reported

    The checkpoint state_dict contains keys that are not used by the model:
pixel_mean
pixel_std
~/anaconda3/envs/densepose/lib/python3.7/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/TensorShape.cpp:2894.)
return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
File "apply_net.py", line 354, in <module>
main()
File "apply_net.py", line 350, in main
args.func(args)
File "apply_net.py", line 106, in execute
cls.execute_on_outputs(context, {"file_name": file_name, "image": img}, outputs)
File "apply_net.py", line 284, in execute_on_outputs
data = extractor(outputs)
File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 152, in __call__
data = extractor(instances, select)
File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 102, in __call__
results = [converter.convert(dpout[i], boxes_xyxy[[i]]) for i in range(len(dpout))]
File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 102, in <listcomp>
results = [converter.convert(dpout[i], boxes_xyxy[[i]]) for i in range(len(dpout))]
File "~/Projects/detectron2/projects/DensePose/densepose/converters/to_chart_result.py", line 65, in convert
predictor_outputs, boxes, *args, **kwargs
File "~/Projects/detectron2/projects/DensePose/densepose/converters/base.py", line 83, in convert
raise KeyError(f"Could not find converter from {instance_type} to {output_type_str}")
KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

    Code Checking

    I found that, in the detectron2/projects/DensePose/densepose/converters/base.py, _lookup_converter function returns None because there are only keys of <class 'densepose.structures.chart.DensePoseChartPredictorOutput'> in cls.registry, how can I modify this file to make the return result correct?

    # the key-values in cls.registry using print function to output
{<class 'densepose.structures.chart.DensePoseChartPredictorOutput'>: <function densepose_chart_predictor_output_to_result_with_confidences at 0x7fb445a66dd0>}
{<class 'densepose.structures.chart.DensePoseChartPredictorOutput'>: <function densepose_chart_predictor_output_to_result_with_confidences at 0x7fb445a66dd0>}

    Environment:

    ----------------------  --------------------------------------------------------------------------------------------------------
sys.platform            linux
Python                  3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21) [GCC 9.4.0]
numpy                   1.21.5
detectron2              0.6 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/detectron2
Compiler                GCC 5.5
CUDA compiler           CUDA 11.3
detectron2 arch flags   7.5
DETECTRON2_ENV_MODULE   <not set>
PyTorch                 1.12.1 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/torch
PyTorch debug build     False
GPU available           Yes
GPU 0                   NVIDIA GeForce GTX 1660 (arch=7.5)
Driver version          470.86
CUDA_HOME               /usr/local/cuda-11.3
Pillow                  9.2.0
torchvision             0.13.1 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/torchvision
torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
fvcore                  0.1.5.post20220512
iopath                  0.1.9
cv2                     4.6.0
----------------------  --------------------------------------------------------------------------------------------------------
PyTorch built with:
- GCC 9.3
- C++ Version: 201402
- Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.3
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.3.2  (built against CUDA 11.5)
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
github-actions[bot] commented 2 years ago

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template. The following information is missing: "Instructions To Reproduce the Issue and Full Logs";

vnghia commented 2 years ago

Are you able to solve this bug ? I meet the same bug today :(

nxrcqupt01 commented 1 year ago

Are you able to solve this bug ? I meet the same bug today

kim-sehun commented 9 months ago

anj dlEKsrjf zhemfkrh rhdrogoshKTdj qudtlsrkxdl