-
### System Info
- CPU x86_64
- GPU: L40
- tensorrt_llm: 0.11.0
- CUDA: 12.4
- driver: 535.129.03
- OS: CentOS 7
### Who can help?
When I tried to import tensorrt_llm, it got stuck. Through debuggi…
-
## Description
(A clear and concise description of what the bug is.)
Model artifacts are in the (TRT-LLM) LMI model format:
` aws s3 ls ***
PRE 1/
2024-10-25 14:59:…
-
docker.io/tensorrt_llm/release:latest
-
### System Info
System Information:
CPU architecture: x86_64
CPU/Host memory size: 2.0 TiB
GPU Properties:
GPU name: NVIDIA H100 80GB HBM3
GPU memory size: 80 GB (75016 MiB / 81559…
-
### System Info
x86_64, Debian 11, L4 GPU
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supporte…
-
### System Info
cpu: x86_64
gpu: nvidia a100
### Who can help?
_No response_
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [x] An officially suppo…
-
## Environment
- **GPUs**: 4x NVIDIA A100 (80GB) (nvlink. azure Standard_NC96ads_A100_v4)
- **TensorRT-LLM Version**: 0.15.0.dev2024102200
- **Environment**: Docker container
- **Memory Usage per GPU…
-
Whether GroundingDINO can support TensorRT-LLM multimodal ？
[TensorRT-LLM multimodal ](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/multimodal/README.md)
-
TRT-LLM version: v0.11.0
I'm deploying a bart model with medusa heads, and i notice this issue https://github.com/NVIDIA/TensorRT-LLM/issues/1946, then i adapted my model with follow steps:
```
1…
-
### System Info
GPU: 4090
Tensorrt: 10.3
tensorrt-llm: 0.13.0.dev2024081300
### Who can help?
@Tracin May you please have a look, thank you very much
### Information
- [ ] The official example sc…