open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.77k stars 636 forks source link

win10 C++ SDK #892

Closed aChang146 closed 2 years ago

aChang146 commented 2 years ago

I have the following problem when I call the SDK of C++ on win10:

$ ./object_detection.exe cpu ../../../onnx_infer/0809_test  ../../../onnx_infer/test.jpg 
loading mmdeploy_execution ...
loading mmdeploy_cpu_device ...
loading mmdeploy_graph ...
loading mmdeploy_directory_model ...
[2022-08-12 15:09:56.042] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
loading mmdeploy_transform ...
loading mmdeploy_cpu_transform_impl ...
loading mmdeploy_transform_module ...  
loading mmdeploy_ort_net ...
failed to load library mmdeploy_ort_net
loading mmdeploy_net_module ...        
loading mmdeploy_mmcls ...
loading mmdeploy_mmdet ... 
loading mmdeploy_mmseg ... 
loading mmdeploy_mmocr ... 
loading mmdeploy_mmedit ...
loading mmdeploy_mmpose ...
loading mmdeploy_mmrotate ...
[2022-08-12 15:09:56.208] [mmdeploy] [info] [model.cpp:38] DirectoryModel successfully load model ../../../onnx_infer/0809_test
[2022-08-12 15:09:56.210] [mmdeploy] [error] [net_module.cpp:38] Net backend not found: onnxruntime, available backends: []
[2022-08-12 15:09:56.210] [mmdeploy] [error] [task.cpp:67] error parsing config: {
  "context": {
    "device": "<any>",
    "model": "<any>",
    "stream": "<any>"
  },
  "input": [
    "prep_output"
  ],
  "input_map": {
    "img": "input"
  },
  "module": "Net",
  "name": "maskrcnn",
  "output": [
    "infer_output"
  ],
  "type": "Task"
}
[2022-08-12 15:09:56.212] [mmdeploy] [error] [pipeline.cpp:145] Could not create Task: maskrcnn
[2022-08-12 15:09:56.212] [mmdeploy] [error] [pipeline.cpp:159] error parsing config: unknown (6)
[2022-08-12 15:09:56.212] [mmdeploy] [error] [handle.h:31] Failed to create pipeline, config: {
  "context": {
    "device": "<any>",
    "stream": "<any>"
  },
  "pipeline": {
    "input": [
      "image"
    ],
    "output": [
      "det"
    ],
    "tasks": [
      {
        "input": [
          "image"
        ],
        "name": "mmdetection",
        "output": [
          "det"
        ],
        "params": {
          "model": "<any>"
        },
        "type": "Inference"
      }
    ]
  }
}
[2022-08-12 15:09:56.230] [mmdeploy] [error] [pipeline.cpp:27] exception caught: unknown (6)
failed to create detector, code: 6

I configured the environment on win10 according to the following link:

https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/01-how-to-build/windows.md

After the environment configuration is completed, the following files are included in mmdeploy/build/bin/release/

image_classification.exe
image_restorer.exe
image_segmentation.exe
mmdeploy.dll
mmdeploy_classifier.dll
mmdeploy_common.dll
mmdeploy_core.dll
mmdeploy_cpu_device.dll
mmdeploy_cpu_transform_impl.dll
mmdeploy_detector.dll
mmdeploy_directory_model.dll
mmdeploy_execution.dll
mmdeploy_executor.dll
mmdeploy_graph.dll
mmdeploy_mmcls.dll
mmdeploy_mmdet.dll
mmdeploy_mmedit.dll
mmdeploy_mmocr.dll
mmdeploy_mmpose.dll
mmdeploy_mmrotate.dll
mmdeploy_mmseg.dll
mmdeploy_model.dll
mmdeploy_net_module.dll
mmdeploy_onnxruntime_ops.dll
mmdeploy_opencv_utils.dll
mmdeploy_ort_net.dll
mmdeploy_pipeline.dll
mmdeploy_pose_detector.dll
mmdeploy_python.cp38-win_amd64.pyd
mmdeploy_restorer.dll
mmdeploy_rotated_detector.dll
mmdeploy_segmentor.dll
mmdeploy_text_detector.dll
mmdeploy_text_recognizer.dll
mmdeploy_transform.dll
mmdeploy_transform_module.dll
object_detection.exe
ocr.exe
pose_detection.exe
rotated_object_detection.exe

It seems that I have successfully compiled onnxruntime, but when calling the SDK objcet_detector.exe, the onnxruntime library cannot be found(I have added the installation directory of onnxtruntime( D:\onnxruntime-win-x64-1.11.0\lib)to the path of the system). How to solve this problem?

My anaconda environment is as follows:

# packages in environment at D:\conda\envs\openmmlab:
#
# Name                    Version                   Build  Channel
addict                    2.4.0                    pypi_0    pypi
bzip2                     1.0.8                h8ffe710_4    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
ca-certificates           2022.6.15            h5b45459_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
certifi                   2022.6.15                pypi_0    pypi
charset-normalizer        2.1.0                    pypi_0    pypi
flatbuffers               2.0                      pypi_0    pypi
idna                      3.3                      pypi_0    pypi
libffi                    3.4.2                h8ffe710_5    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
libzlib                   1.2.12               h8ffe710_2    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
mmcv                      1.6.1                    pypi_0    pypi
numpy                     1.23.1                   pypi_0    pypi
onnxruntime               1.8.1                    pypi_0    pypi
opencv-python             4.6.0.66                 pypi_0    pypi
openssl                   3.0.5                h8ffe710_1    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
packaging                 21.3                     pypi_0    pypi
pillow                    9.2.0                    pypi_0    pypi
pip                       22.2.2             pyhd8ed1ab_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
protobuf                  4.21.5                   pypi_0    pypi
pyparsing                 3.0.9                    pypi_0    pypi
python                    3.8.13          hcf16a7b_0_cpython    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
python_abi                3.8                      2_cp38    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
pyyaml                    6.0                      pypi_0    pypi
regex                     2022.7.25                pypi_0    pypi
requests                  2.28.1                   pypi_0    pypi
setuptools                63.4.3           py38haa244fe_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
sqlite                    3.39.2               h8ffe710_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
tk                        8.6.12               h8ffe710_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
torch                     1.12.1                   pypi_0    pypi
torchaudio                0.12.1                   pypi_0    pypi
torchvision               0.13.1                   pypi_0    pypi
typing-extensions         4.3.0                    pypi_0    pypi
ucrt                      10.0.20348.0         h57928b3_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
urllib3                   1.26.11                  pypi_0    pypi
vc                        14.2                 hb210afc_6    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
xz                        5.2.5                h62dcd97_1    https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge
yapf                      0.32.0                   pypi_0    pypi
irexyc commented 2 years ago

According to the log failed to load library mmdeploy_ort_net, the exe loads mmdeploy_ort_net.dll failed.

You may try to check if is it is due to misuse of onnxruntime.dll https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/faq.md#onnx-runtime

aChang146 commented 2 years ago

According to the log failed to load library mmdeploy_ort_net, the exe loads mmdeploy_ort_net.dll failed.

You may try to check if is it is due to misuse of onnxruntime.dll https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/faq.md#onnx-runtime

I tried copying the onnxruntime.dll to the sibling directory of mmdeploy_onnxruntime_ops.dll as described above and it still failed,The error log is as follows:

 cmake ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.32.31332.0
-- The CXX compiler identification is MSVC 19.32.31332.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: D:/vs2019/1/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: D:/vs2019/1/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenCV ARCH: x64
-- OpenCV RUNTIME: vc15
-- OpenCV STATIC: OFF
-- Found OpenCV: D:/Opencv4.5.5/opencv/build (found version "4.5.5")
-- Found OpenCV 4.5.5 in D:/Opencv4.5.5/opencv/build/x64/vc15/lib
-- You might need to add D:\Opencv4.5.5\opencv\build\x64\vc15\bin to your PATH to be able to run your applications.
CMake Error at D:/mmdeploy/build/MMDeployConfig.cmake:12 (include):
  include could not find requested file:

    D:/mmdeploy/build/MMDeployTargets.cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (find_package)

CMake Error at D:/mmdeploy/build/MMDeployConfig.cmake:45 (find_package):
  By not providing "FindONNXRUNTIME.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "ONNXRUNTIME", but CMake did not find one.

  Could not find a package configuration file provided by "ONNXRUNTIME" with
  any of the following names:

    ONNXRUNTIMEConfig.cmake
    onnxruntime-config.cmake

  Add the installation prefix of "ONNXRUNTIME" to CMAKE_PREFIX_PATH or set
  "ONNXRUNTIME_DIR" to a directory containing one of the above files.  If
  "ONNXRUNTIME" provides a separate development package or SDK, be sure it
  has been installed.
Call Stack (most recent call first):
  CMakeLists.txt:6 (find_package)

-- Configuring incomplete, errors occurred!
See also "D:/mmdeploy/onnx_infer/build/CMakeFiles/CMakeOutput.log".

CMakeLists.txts is as follows:

cmake_minimum_required(VERSION 3.14)
project(mmdeploy-example)
find_package(OpenCV REQUIRED)
set(MMDEPLOY "D:\\mmdeploy\\build\\install\\bin")
set(MMDeploy_DIR "D:\\mmdeploy\\build")
find_package(MMDeploy)

# Add OpenCV headers location to your include paths
include_directories(${OpenCV_INCLUDE_DIRS})
include_directories(${MMDEPLOY}/include)
add_executable(detect onnx_infer.cpp)
target_link_libraries(detect ${OpenCV_LIBS} mmdeploy)
irexyc commented 2 years ago

I tried copying the onnxruntime.dll to the sibling directory of mmdeploy_onnxruntime_ops.dll as described above and it still failed,The error log is as follows:

How did you compile it successfully for the first time? As your original problem is failed to load library mmdeploy_ort_net other than build error

You can't only copy to onnxruntime.dll but copy all dll files in D:\onnxruntime-win-x64-1.11.0\lib folder. If mmdeploy_core.dll located in D:\mydll, and you add D:\mydll to the PATH, you also need to copy onnx dlls to D:\mydll.

aChang146 commented 2 years ago

Thanks,I have solved this problem by changing the CMakeLists.txt as follows:

cmake_minimum_required(VERSION 3.14)
project(mmdeploy-example)
find_package(OpenCV REQUIRED)
set(MMDEPLOY "D:\\mmdeploy\\build\\install\\bin")
set(MMDeploy_DIR "D:\\mmdeploy\\build\\install\\lib\\cmake\\MMDeploy")
find_package(MMDeploy)

# Add OpenCV headers location to your include paths
include_directories(${OpenCV_INCLUDE_DIRS})
include_directories(${MMDEPLOY}/include)
add_executable(detect onnx_infer.cpp)
target_link_libraries(detect ${OpenCV_LIBS} mmdeploy)
aChang146 commented 2 years ago

I tried copying the onnxruntime.dll to the sibling directory of mmdeploy_onnxruntime_ops.dll as described above and it still failed,The error log is as follows:

How did you compile it successfully for the first time? As your original problem is failed to load library mmdeploy_ort_net other than build error

I recompiled the mmdeploy library according to the following tutorial https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/02-how-to-run/prebuilt_package_windows.md, and then copied the **onnxruntime.dll**to the same directory of **mmdeploy.dll**, after this operation, I can execute build/bin/Release/object_detection.exe, for the second manual compilation problem, I took a CMakeLists.txt that looks like this:

cmake_minimum_required(VERSION 3.14)
project(mmdeploy-example)
find_package(OpenCV REQUIRED)
set(MMDEPLOY "D:\\mmdeploy\\build\\install\\bin")
set(MMDeploy_DIR "D:\\mmdeploy\\build\\install\\lib\\cmake\\MMDeploy")
find_package(MMDeploy)

# Add OpenCV headers location to your include paths
include_directories(${OpenCV_INCLUDE_DIRS})
include_directories(${MMDEPLOY}/include)
add_executable(detect onnx_infer.cpp)
target_link_libraries(detect ${OpenCV_LIBS} mmdeploy)

Executed the following command:

mkdir build&&cd build
cmake ..

The execution result of this command is as follows:

cmake ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.32.31332.0
-- The CXX compiler identification is MSVC 19.32.31332.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: D:/vs2019/1/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: D:/vs2019/1/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenCV ARCH: x64
-- OpenCV RUNTIME: vc15
-- OpenCV STATIC: OFF
-- Found OpenCV: D:/Opencv4.5.5/opencv/build (found version "4.5.5") 
-- Found OpenCV 4.5.5 in D:/Opencv4.5.5/opencv/build/x64/vc15/lib    
-- You might need to add D:\Opencv4.5.5\opencv\build\x64\vc15\bin to your PATH to be able to run your applications.
-- OpenCV ARCH: x64
-- OpenCV RUNTIME: vc15
-- OpenCV STATIC: OFF
-- Found OpenCV 4.5.5 in D:/Opencv4.5.5/opencv/build/x64/vc15/lib
-- You might need to add D:\Opencv4.5.5\opencv\build\x64\vc15\bin to your PATH to be able to run your applications.
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: D:/mmdeploy/cmake_build_test/build

Then,I executed the following command:

cmake --build . --config Release
用于 .NET Framework 的 Microsoft (R) 生成引擎版本 17.2.1+52cd2da31
版权所有(C) Microsoft Corporation。保留所有权利。

  Checking Build System
  Building Custom Rule D:/mmdeploy/cmake_build_test/CMakeLists.txt
  onnx_infer.cpp
  detect.vcxproj -> D:\mmdeploy\cmake_build_test\build\Release\detect.exe
  Building Custom Rule D:/mmdeploy/cmake_build_test/CMakeLists.txt

After that, in order to ensure that detect.exe can be executed correctly, I added the directory of the dynamic link library that detect.exe may use to the PATH, and then the program can run correctly