search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.35k stars 1.49k forks source link

Error building Triton Docker image in CPU-Only mode with TensorFlow2 backend #7732

Open PierreCarceller opened 3 weeks ago

PierreCarceller commented 3 weeks ago

Description I want to build a docker image of triton in CPU-ONLY mode. I followed this section of the doc to get there:

I executed :``` bash git clone https://github.com/triton-inference-server/server.git cd server git checkout v2.50.0

python3 build.py --backend tensorflow2 --repoagent checksum --container-version 24.09 -v --extra-backend-cmake-arg=tensorflow2:TRITON_TENSORFLOW_INSTALL_EXTRA_DEPS=ON 


and I got this : 

``` bash
/home/pcarceller/triton_docker/server/build.py:1229: SyntaxWarning: invalid escape sequence '\;'
  df += """
/home/pcarceller/triton_docker/server/build.py:1517: SyntaxWarning: invalid escape sequence '\o'
  df = """
/home/pcarceller/triton_docker/server/build.py:1714: SyntaxWarning: invalid escape sequence '\p'
  runargs += ["-v", "\\\\.\pipe\docker_engine:\\\\.\pipe\docker_engine"]
Building Triton Inference Server
platform linux
machine x86_64
version 2.50.0
build dir /home/pcarceller/triton_docker/server/build
install dir None
cmake dir None
default repo-tag: r24.09
container version 24.09
upstream container version 24.09
backend "tensorflow" at tag/branch "r24.09"
repoagent "checksum" at tag/branch "r24.09"
error: --extra-backend-cmake-arg specifies backend "tensorflow2" which is not included in build

But if I change the order a little like this it works : 

python3 build.py --backend tensorflow --repoagent checksum --container-version 24.09 -v --extra-backend-cmake-arg=tensorflow:TRITON_TENSORFLOW_INSTALL_EXTRA_DEPS=ON

Did I make a mistake or did the doc give the wrong information?

rmccorm4 commented 5 days ago

Hi @PierreCarceller,

I believe the issue lies in the tensorflow2 used instead of tensorflow in --extra-backend-cmake-arg=tensorflow2:TRITON_TENSORFLOW_INSTALL_EXTRA_DEPS=ON

When you run --backend tensorflow2 (we internally translate that to tensorflow), but the --extra-backend-cmake-arg looks like it doesn't have the same logic to handle the user providing "tensorflow2", so you should use tensorflow instead of tensorflow2.