Closed nv-kmcgill53 closed 1 month ago
Can you post the Docker command I can use to reproduce the error?
Hi @hcho3, we use the below steps to build FIL backend:
git clone --recursive --single-branch --depth=1 -b main https://github.com/triton-inference-server/fil_backend.git fil
cd fil
mkdir build && cd build
cmake -DTRT_VERSION=10.1.0.27+cuda12.4.1.003 -DCMAKE_TOOLCHAIN_FILE= -DVCPKG_TARGET_TRIPLET= -DTRITON_FIL_DOCKER_BUILD:BOOL=ON -DTRITON_BUILD_CONTAINER=nvcr.io/nvidia/tritonserver:24.06-py3-min -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=`pwd`/install -DTRITON_REPO_ORGANIZATION:STRING=https://github.com/triton-inference-server -DTRITON_COMMON_REPO_TAG:STRING=main -DTRITON_CORE_REPO_TAG:STRING=main -DTRITON_BACKEND_REPO_TAG:STRING=main -DTRITON_ENABLE_GPU:BOOL=ON -DTRITON_ENABLE_MALI_GPU:BOOL=OFF -DTRITON_ENABLE_STATS:BOOL=ON -DTRITON_ENABLE_METRICS:BOOL=ON -DTRITON_ENABLE_MEMORY_TRACKER:BOOL=ON ..
cmake --build . --config Release -j256 -v -t install
@krishung5 Did you apply the patch 0001-Allow-predicting-with-FP32-input-and-FP64-models.patch
to _deps/treelite-src
?
I don't think we did. Is the patch required to build on main or r24.07 branch? Let me try that meanwhile.
Yes. The patch is required. Does your build system not use https://github.com/triton-inference-server/fil_backend/blob/main/build.sh from this repo? The build.sh
should have picked up the patch automatically.
Running this command
cmake -DTRT_VERSION=10.1.0.27+cuda12.4.1.003 -DCMAKE_TOOLCHAIN_FILE= -DVCPKG_TARGET_TRIPLET= -DTRITON_FIL_DOCKER_BUILD:BOOL=ON -DTRITON_BUILD_CONTAINER=nvcr.io/nvidia/tritonserver:24.06-py3-min -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=`pwd`/install -DTRITON_REPO_ORGANIZATION:STRING=https://github.com/triton-inference-server -DTRITON_COMMON_REPO_TAG:STRING=main -DTRITON_CORE_REPO_TAG:STRING=main -DTRITON_BACKEND_REPO_TAG:STRING=main -DTRITON_ENABLE_GPU:BOOL=ON -DTRITON_ENABLE_MALI_GPU:BOOL=OFF -DTRITON_ENABLE_STATS:BOOL=ON -DTRITON_ENABLE_METRICS:BOOL=ON -DTRITON_ENABLE_MEMORY_TRACKER:BOOL=ON ..
invokes the Docker build with ops/Dockerfile
, which already applies the patch automatically. So no need to manually apply the patch again.
Just tried the build command (cmake -DTRT_VERSION=...
) on my end. I can't reproduce the error at all. Let me clean the Docker cache and try again.
@hcho3 Confirmed that I was able to build fil_backend with the fix. Thanks for the quick fix! Could we bring this patch to the release branch as well?
Sure, let me bring it to the release branch.
Triton-Inference-Server is seeing an error when attempting to compile the fil_backend. Below is the error given by cmake:
This failed our nightly build and looks to be related to the following change: https://github.com/triton-inference-server/fil_backend/pull/394
Let us know of any more information you need to troubleshoot, and thanks in advance for the help!