-
**Problem**
TensorRT-LLM is one of the biggest features in jan. As the docker environments usually include all kind of stuff to run the software with it's full set of features, why not include it too…
-
## Bug Description
The example of pytorch tensorrt cannot generate images correctly. There were no issues with version v2.3.0, but version v2.4.0 cannot generate images correctly.
https://git…
-
### Checklist
- [X] I've read the [contribution guidelines](https://github.com/autowarefoundation/autoware/blob/main/CONTRIBUTING.md).
- [X] I've searched other issues and no duplicate issues were…
-
### System Info
A10
tensorrt-cu12-10.2.0.post1
tensorrt-cu12-bindings-10.2.0.post1
tensorrt-cu12-libs-10.2.0.post1
tensorrt_llm-0.12.0.dev2024072300
python==3.10
### Who can help?
@Tracin
…
-
### System Info
GPU: NVIDIA T4 * 4
Driver Version: 550.54.15
CUDA: 12.4
Image: nvcr.io/nvidia/tritonserver:24.07-trtllm-python-py3
TensorRT-LLM version: 0.11.0
### Who can help?
No response…
-
### Describe the issue
Inference results are outputting abnormally when using YOLOv7 models with TensorRT EP.
We have confirmed that the results are normal when using CPU and CUDA.
The issue wa…
-
## Goal
- We should have a model folder that is able to handle different models
- Built-in models (e.g. `janhq/llama3:7b-tensorrt-llm`)
- Huggingface GGUF repos with multiple quants (e.g. `ba…
-
### System Info
A100-PCIe-40GB
Tensorrt-LLM-verison:0.12.0
### Who can help?
@sunnyqgg
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [x] An offi…
-
### System Info
- CPU x86_64 (intel i9)
- 128G memory (RAM)
- GPU: 1 x RTXA6000
- Libraries:
- TensorRT-LLM 0.12.0 (stable)
- TensorRT 10.3.0
- transformers 4.42.4
- CUDA vers…
-
使用ProgramEntrance_1.py导出带qdq节点的onnx模型,速度比fp16的tensorrt模型还要慢是什么情况。