search

triton-inference-server / paddlepaddle_backend

BSD 3-Clause "New" or "Revised" License
32 stars 6 forks source link

Error when building backend with paddle 2.3 #13

Closed thomasyugit closed 1 year ago

thomasyugit commented 2 years ago 

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 22.08 (build 42766143)
Triton Server Version 2.25.0

Copyright (c) 2018-2022, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
   Use the NVIDIA Container Toolkit to start this container with GPU support; see
   https://docs.nvidia.com/datacenter/cloud-native/ .

Get:1 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  InRelease [1581 B]
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:4 https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64  Packages [705 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [27.5 kB]
Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2183 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:8 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [916 kB]
Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1556 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1214 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2650 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [30.2 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1671 kB]
Get:19 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [27.4 kB]
Get:20 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [55.2 kB]
Fetched 24.5 MB in 13s (1862 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  rapidjson-dev
0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded.
Need to get 95.0 kB of archives.
After this operation, 636 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 rapidjson-dev all 1.1.0+dfsg2-5ubuntu1 [95.0 kB]
Fetched 95.0 kB in 2s (54.9 kB/s)
Selecting previously unselected package rapidjson-dev.
(Reading database ... 24861 files and directories currently installed.)
Preparing to unpack .../rapidjson-dev_1.1.0+dfsg2-5ubuntu1_all.deb ...
Unpacking rapidjson-dev (1.1.0+dfsg2-5ubuntu1) ...
Setting up rapidjson-dev (1.1.0+dfsg2-5ubuntu1) ...
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- RapidJSON found. Headers: /usr/include
-- RapidJSON found. Headers: /usr/include
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found CUDAToolkit: /usr/local/cuda/include (found version "11.7.99")
-- Found CUDA: /usr/local/cuda (found version "11.7")
-- Using CUDA 11.7
-- Configuring done
-- Generating done
-- Build files have been written to: /workspace/paddle_backend/build
Scanning dependencies of target triton-common-async-work-queue
[  3%] Building NVCC (Device) object _deps/repo-backend-build/CMakeFiles/kernel-library-new.dir/src/kernel-library-new_generated_kernel.cu.o
Scanning dependencies of target triton-common-table-printer
Scanning dependencies of target triton-common-thread-pool
Scanning dependencies of target triton-common-error
Scanning dependencies of target triton-core-serverstub
Scanning dependencies of target triton-common-logging
[ 11%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-async-work-queue.dir/src/async_work_queue.cc.o
[ 11%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-async-work-queue.dir/src/error.cc.o
[ 15%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-thread-pool.dir/src/thread_pool.cc.o
[ 19%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-table-printer.dir/src/table_printer.cc.o
[ 23%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-error.dir/src/error.cc.o
[ 26%] Building CXX object _deps/repo-core-build/CMakeFiles/triton-core-serverstub.dir/src/tritonserver_stub.cc.o
[ 30%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-logging.dir/src/logging.cc.o
[ 34%] Linking CXX shared library libtritonserver.so
[ 34%] Built target triton-core-serverstub
[ 38%] Building CXX object _deps/repo-common-build/CMakeFiles/triton-common-async-work-queue.dir/src/thread_pool.cc.o
[ 42%] Linking CXX static library libtritoncommonerror.a
[ 42%] Built target triton-common-error
[ 46%] Linking CXX static library libtritoncommonlogging.a
[ 50%] Linking CXX static library libtritonthreadpool.a
[ 50%] Built target triton-common-logging
[ 50%] Built target triton-common-thread-pool
[ 53%] Linking CXX static library libtritonasyncworkqueue.a
[ 53%] Built target triton-common-async-work-queue
[ 57%] Linking CXX static library libtritontableprinter.a
[ 57%] Built target triton-common-table-printer
Scanning dependencies of target kernel-library-new
[ 61%] Linking CXX static library libkernel-library-new.a
[ 61%] Built target kernel-library-new
Scanning dependencies of target triton-backend-utils
[ 65%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_input_collector.cc.o
[ 69%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_common.cc.o
[ 73%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_output_responder.cc.o
[ 80%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_memory.cc.o
[ 80%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_model.cc.o
[ 84%] Building CXX object _deps/repo-backend-build/CMakeFiles/triton-backend-utils.dir/src/backend_model_instance.cc.o
[ 88%] Linking CXX static library libtritonbackendutils.a
[ 88%] Built target triton-backend-utils
Scanning dependencies of target triton-paddle-backend
[ 92%] Building CXX object CMakeFiles/triton-paddle-backend.dir/src/paddle_backend_utils.cc.o
[ 96%] Building CXX object CMakeFiles/triton-paddle-backend.dir/src/paddle.cc.o
/workspace/paddle_backend/src/paddle.cc: In member function 'void ModelImpl::CollectShapeRun(paddle_infer::Predictor*, const std::map<std::__cxx11::basic_string<char>, std::vector<int> >&)':
/workspace/paddle_backend/src/paddle.cc:78:32: error: 'class paddle_infer::Predictor' has no member named 'GetInputTypes'; did you mean 'GetInputNames'?
   78 |   auto input_type = predictor->GetInputTypes();
      |                                ^~~~~~~~~~~~~
      |                                GetInputNames
/workspace/paddle_backend/src/paddle.cc: In constructor 'ModelImpl::ModelImpl(const char*, const char*, TRITONPADDLE_Config*, int32_t, cudaStream_t)':
/workspace/paddle_backend/src/paddle.cc:177:23: error: 'struct paddle::AnalysisConfig' has no member named 'SetExecStream'
  177 |     analysis_config_->SetExecStream((void*)stream);
      |                       ^~~~~~~~~~~~~
/workspace/paddle_backend/src/paddle.cc:198:27: error: 'struct paddle::AnalysisConfig' has no member named 'EnableVarseqlen'
  198 |         analysis_config_->EnableVarseqlen();
      |                           ^~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/triton-paddle-backend.dir/build.make:82: CMakeFiles/triton-paddle-backend.dir/src/paddle.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:170: CMakeFiles/triton-paddle-backend.dir/all] Error 2
make: *** [Makefile:149: all] Error 2

It looks like the compiled paddle inference lib does not have these member functions. And I found that build_paddle.sh build on paddle 2.3

https://github.com/triton-inference-server/paddlepaddle_backend/blob/6c1c935e9978f5606082cfb372cc74948bcb6e7a/paddle-lib/Dockerfile#L48

which is weird, because GetInputTypes was implemented in release/2.4, not in release/2.3

https://github.com/PaddlePaddle/Paddle/blob/release/2.4/paddle/fluid/inference/api/paddle_api.h https://github.com/PaddlePaddle/Paddle/blob/release/2.3/paddle/fluid/inference/api/paddle_api.h

Since paddle 2.4 has not been released, I do not know if compiling backend with dev version is recommended. @heliqi could you help me deal with it?

heliqi commented 1 year ago

Sorry, I missed the message. Paddle 2.4 has been released and we also recommend the develop version of paddle.