-
Hi,
We are trying to use TensorRT to speed up inference. In particular, we are using DetectorSOLOv2, and installed a version of paddle-paddle-gpu compiled with TensorRT. However, the inference speed …
-
### Search before asking
- [ ] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussion…
-
### System Info
- CPU architecture :× 86 64 , aarch64)
- GPU properties - GPU name NVIDIA A800
- GPU memory size (80G)f known)
- Libraries 一 TensorRT-LLM tag : v0.10.0 , tensorrtllm-backend ta…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
## Description
(A clear and concise description of what the bug is.)
I'm am building the DJL-Serving TensorRT-LLM LMI inference container from scratch, and deploying on Sagemaker Endpoints for Zep…
-
How to load multiple Lora weights and multiple text inputs to inference?
Currently, only single Lora weights and input tokens are supported as inputs. How to support multiple Lora weights and input t…
-
## Description
When I tried to convert my onnx files to the tensorrt engine using tensorrt, the entire network was converted to a huge layer. However, my original model was very large and complex, …
-
https://github.com/YuzhouPeng/unet-tensorrt/blob/a7a6d81ddf1ed553a87d1bdbd83075c6b892dacb/unet.cpp#L363
why using the sigmoid?
-
### System Info
- CPU architecture: x86_64
- GPU name: NVIDIA A40, 46GB
- TensorRT-LLM: v0.9.0
- Os: Ubuntu 20.04
- Nvidia Driver: 535.54.03, Cuda: 12.2
### Who can help?
@kaiyux @byshiue…
-
I'm following build instruction and I get this build error:
```
stiv@nano:~/TensorRT-Yolov3/build$ git submodule update --init --recursive
Submodule 'tensorRTWrapper' (https://github.com/lewes636…