When trying to use this on TritonSever, i have to install setuptools.

NVIDIA / TensorRT-Model-Optimizer

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.

https://nvidia.github.io/TensorRT-Model-Optimizer

Other

458 stars 30 forks source link

When trying to use this on TritonSever, i have to install setuptools. #66

Open dongs0104 opened 1 month ago

dongs0104 commented 1 month ago

modelopt is already installed on tritonserver:24.08-trtllm but setuptools is not installed, so i get error when i use it

please add dependency

cjluo-omniml commented 2 weeks ago

Are you able to install setuptools as a separate step?