-
Hi,
I noticed there is no slack, discord or irc channel for tensorrt - which could offload some future tickets by discussing things in the channel - so I created one.
I hope its ok to advertise …
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and f…
-
Bug Description:
When the Triton Inference Server experiences high traffic, it appears to freeze and stops processing incoming requests. During this time, the GPU utilization reaches 100% and stays s…
-
I would like to use techniques such as Multi-instance Support supported by the tensorrt-llm backend. In the documentation, I can see that multiple models are served using modes like Leader mode and …
-
## 我的代码:
```
class Predictor:
def __init__(self, args):
self.args = args
self.cfg = DeployConfig(args.config, args.vertical_screen)
self.compose = T.Compose(self.cfg.…
-
### System Info
I am working on the benchmarking suite in vLLM team, and now trying to run TensorRT-LLM for comparison. I am relying on this github repo (https://github.com/neuralmagic/tensorrt-demo)…
-
### System Info
- CPU Architecture x86_64
- GPU: NVIDIA T4 * 4 (AWS g4dn.12xLarge)
- TensorRT-LLM v0.10.0
### Who can help?
[QiJune](https://github.com/QiJune)
@byshiue
### Information
…
-
### System Info
CPU Architecture x86
A100 40GB
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially…
jlewi updated
1 month ago
-
### System Info
- CPU architecture : x86_64
- CPU/Host memory size : 32 GB
- GPU name L4 at g2-standard-8 (GCP)
- GPU memory size 24GB
- TensorRT-LLM branch or tag (e.g., main, v0.10.0)
- Nvi…
-
### System Info
Hi,
I noticed there is no slack, discord or irc channel for tensorrt - which could offload some future tickets by discussing things in the channel - so I created one.
I hope its…