-
### System Info
- nvidia:535.129.03
- cuda_version:12.4
- GPU:L40S
- OS:Ubuntu 22.04.4 LTS(docker)
- tensorrt-llm: 0.11.0.dev2024060400
### Who can help?
_No response_
### Information
…
-
Thank you for your excellent work! :satisfied: :satisfied: :satisfied:
Recently, I have been trying to use TensorRT to accelerate Depth Anything on Jetson Orin NX. However, I found that the infere…
-
On my system, I have enough VRAM (72 GB) to run Llama-3-70B in 4-bit or 8-bit precision. However, I am unable to quantize this model to either 4-bit or 8-bit precision using the scripts in TensorRT-LL…
-
Thank you for your excellent work! :satisfied: :satisfied: :satisfied:
Recently, I have been trying to use TensorRT to accelerate Depth Anything on Jetson Orin NX. However, I found that the infere…
-
In TensorRT 10.1, if you try to use build_engine, the following error occurs.
[AttributeError: 'tensorrt_bindings.tensorrt.Builder' object has no attribute 'build_engine']
How do I use build_engin…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
As mentioned in the title, when trying to convert Loras into TensorRT format via TensorRT tab, in the dropdown where one is supposed to select the Lora, only Loras from the models/Lora directory are a…
-
## Description
`pip install --extra-index-url https://pypi.nvidia.com tensorrt==10.0.1` now installs tensorrt 10.1 instead of 10.0.1 and fails to deserialize engines built with 10.0.1
## Steps…
-
**Project description**
Project cudaPackages.tensorrt version is already 10.1.0.27, but in the nixpkgs it is still 8.6.1.6 (which is broken by the way).
**Metadata**
* homepage URL: https://…
-
## Description
## Environment
**TensorRT Version**:
**NVIDIA GPU**:
**NVIDIA Driver Version**:
**CUDA Version**:
**CUDNN Version**:
Operating System:
Python Version…