-
## Description
I tired running the bertQA sample in Jetson Orin nano with jetpack 6.1
I used Bert Base, because Bert Large kills itself when building the engine(may be because of memory issue).
```
[…
-
## Description
I tried to convert my model to the engine using tensorrt. My model has a lot of cross-attention structures, similar to Transformer. First of all, with precision fp32, it runs without…
-
I use modelopt QAT my model:
```
import modelopt.torch.quantization as mtq
# Select quantization config
config = mtq.INT8_DEFAULT_CFG
# Define forward loop for calibration
def forward_loop(model):
…
-
Hi, I use the trtexec cmd tool to generate a plan trt model, Can I load it by TensorRT Loader node?
```
/usr/src/tensorrt/bin/trtexec --builderOptimizationLevel=4 --stronglyTyped --onnx=./backbone.o…
-
when I convert the onnx model to TRT model, get this error below.
can you tell me how to fix it.
-
PS C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Upscaler-Tensorrt> python export_trt.py
Traceback (most recent call last):
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyU…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussi…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and found no similar bug report.
### Ultralytics YOLO Component
Pred…
-
## Description
(A clear and concise description of what the bug is.)
Model artifacts are in the (TRT-LLM) LMI model format:
` aws s3 ls ***
PRE 1/
2024-10-25 14:59:…
-
### System Info
- CPU x86_64 (intel i9)
- 128G memory (RAM)
- GPU: 1 x RTXA6000
- Libraries:
- TensorRT-LLM 0.12.0 (stable)
- TensorRT 10.3.0
- transformers 4.42.4
- CUDA vers…