search

NVIDIA / TensorRT-Model-Optimizer

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.
https://nvidia.github.io/TensorRT-Model-Optimizer
Other
576 stars 43 forks source link

Has a plan to support of Jetson AGX Orig?? #7

Closed Vaderpucong closed 5 months ago

Vaderpucong commented 6 months ago 
### Tasks
Vaderpucong commented 6 months ago

jetson has a specific version of pytorch that is not compatible with the current TensorRT-Model-Optimizer. If TensorRT-Model-Optimizer is installed, this version is removed, causing the GPU to be unrecognized: image

kevalmorabia97 commented 6 months ago

model optimizer supports torch>=1.11. What version are you using?

Vaderpucong commented 6 months ago

model optimizer supports torch>=1.11. What version are you using?

torch-2.3.0a0+ebedce2.nv24.02-cp310-cp310-linux_aarch64.whl

This is a Specific version for Jetson AGX Orin

kevalmorabia97 commented 6 months ago

Can you check the full installation logs if something else is causing this to be uninstalled. ModelOpt should be compatible with the 2.3.0a0... torch version