search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.39k stars 1.49k forks source link

build: RHEL8 EA2 Backends #7568

Closed fpetrini15 closed 3 months ago

fpetrini15 commented 3 months ago

Goal: Support the PyTorch, TensorRT, and TensorFlow backends on RHEL8 systems. Importantly, these changes will disable optional PyTorch optional features such as TRITON_PYTORCH_ENABLE_TORCHTRT and TRITON_ENABLE_NVTX, which are not currently supported for the RHEL8 build.

PyTorch Backend: https://github.com/triton-inference-server/pytorch_backend/pull/137 TensorRT Backend: https://github.com/triton-inference-server/tensorrt_backend/pull/98 TensorFlow: https://github.com/triton-inference-server/tensorflow_backend/pull/105 ORT Backend: https://github.com/triton-inference-server/onnxruntime_backend/pull/266