[Tritonserver with FIL backend] Build failure when installing raft

[nikhil-sk]

Hi, I'm trying to build a Triton container with the FIL backend (with version 22.09), and I'm running into the failure: Build command:

./build.py --enable-logging --enable-stats --enable-tracing --enable-metrics --enable-gpu-metrics --enable-gpu --no-container-interactive --endpoint=http --endpoint=grpc --endpoint=sagemaker --repo-tag=common:$RELEASE_TAG --repo-tag=core:$RELEASE_TAG --repo-tag=backend:$RELEASE_TAG --repo-tag=thirdparty:$RELEASE_TAG --backend=ensemble:$RELEASE_TAG --backend=tensorrt:$RELEASE_TAG --backend=identity:$RELEASE_TAG --backend=repeat:$RELEASE_TAG --backend=square:$RELEASE_TAG --backend=onnxruntime:$RELEASE_TAG --backend=pytorch:$RELEASE_TAG --backend=tensorflow1:$RELEASE_TAG --backend=tensorflow2:$RELEASE_TAG --backend=python:$RELEASE_TAG --backend=dali:$RELEASE_TAG --backend=fil:$RELEASE_TAG --backend=fastertransformer:main --repoagent=checksum:$RELEASE_TAG

here $RELEASE_TAG=r22.09

#32 19.14 -- Check if compiler accepts -pthread - yes
#32 19.14 -- CUML: RAFT pinned tag found: b88939402d63682458fccf688acae1240fcfa67f. Cloning raft locally.
#32 19.14 -- CUML: raft FIND_PACKAGE_ARGUMENTS COMPONENTS 
#32 19.14 -- CPM: cuml: adding package raft@22.10.00 (b88939402d63682458fccf688acae1240fcfa67f)
#32 20.12 [1/9] Creating directories for 'raft-populate'
#32 20.12 [1/9] Performing download step (git clone) for 'raft-populate'
#32 20.12 Cloning into 'raft-src'...
#32 20.12 fatal: reference is not a tree: b88939402d63682458fccf688acae1240fcfa67f
#32 20.12 CMake Error at raft-subbuild/raft-populate-prefix/tmp/raft-populate-gitclone.cmake:40 (message):
#32 20.12   Failed to checkout tag: 'b88939402d63682458fccf688acae1240fcfa67f'
#32 20.12 
#32 20.12 
#32 20.12 FAILED: raft-populate-prefix/src/raft-populate-stamp/raft-populate-download /rapids_triton/build/_deps/raft-subbuild/raft-populate-prefix/src/raft-populate-stamp/raft-populate-download 
#32 20.12 cd /rapids_triton/build/_deps && /conda/dev/bin/cmake -P /rapids_triton/build/_deps/raft-subbuild/raft-populate-prefix/tmp/raft-populate-gitclone.cmake && /conda/dev/bin/cmake -E touch /rapids_triton/build/_deps/raft-subbuild/raft-populate-prefix/src/raft-populate-stamp/raft-populate-download
#32 20.12 ninja: build stopped: subcommand failed.
#32 20.12 
#32 20.12 CMake Error at /conda/dev/share/cmake-3.22/Modules/FetchContent.cmake:1087 (message):
#32 20.12   Build step for raft failed: 1
#32 20.12 Call Stack (most recent call first):
#32 20.12   /conda/dev/share/cmake-3.22/Modules/FetchContent.cmake:1216:EVAL:2 (__FetchContent_directPopulate)
#32 20.12   /conda/dev/share/cmake-3.22/Modules/FetchContent.cmake:1216 (cmake_language)
#32 20.12   build/cmake/CPM_0.35.0.cmake:885 (FetchContent_Populate)
#32 20.12   build/cmake/CPM_0.35.0.cmake:685 (cpm_fetch_package)
#32 20.12   build/cmake/CPM_0.35.0.cmake:261 (CPMAddPackage)
#32 20.12   build/_deps/rapids-cmake-src/rapids-cmake/cpm/find.cmake:152 (CPMFindPackage)
#32 20.12   build/_deps/cuml-src/cpp/cmake/thirdparty/get_raft.cmake:54 (rapids_cpm_find)
#32 20.12   build/_deps/cuml-src/cpp/cmake/thirdparty/get_raft.cmake:83 (find_and_configure_raft)
#32 20.12   build/_deps/cuml-src/cpp/CMakeLists.txt:201 (include)
#32 20.12 
#32 20.12 
#32 20.12 -- Configuring incomplete, errors occurred!
#32 20.12 See also "/rapids_triton/build/CMakeFiles/CMakeOutput.log".
#32 20.12 See also "/rapids_triton/build/CMakeFiles/CMakeError.log".

There seems to be a redirection to the wphicks fork and something breaks along the way, at least for r22.09 branch. Request you to fix it.

Thank you.

[wphicks]

This was resolved for 22.10, and I've created a backport PR here. It can be merged once it clears CI.

[wphicks]

The backport has now been merged. Please let us know if you have any other issues with the build. Thanks!