-
tensorflow-gpu1.14
CUDA10.1
GPU gtx1080
my_test.json
{
"model_config": {
"model_name": "ssd_resnet_50_fpn_coco",
"input_dir": "/home/liujt/software/tensorrt/data",
"batch_size…
-
I get this error in Tesla T4, using the following code:
```
[TensorRT] ERROR: No non-int8 implementation of layer [CONVOLUTION #1]
CUDA_VISIBLE_DEVICES=0 python infer.py --m resnet18 --load_ckpt …
KeyKy updated
3 years ago
-
### System Info
- Ubuntu
- GPU A100 / 3090 RTX
- docker nvcr.io/nvidia/tritonserver:24.02-trtllm-python-py3
- Python tensorrt-llm package (version 0.9.0.dev2024030500) installed in the docker im…
-
## Description
almost same params
Even use int8 , it can't save more memory and slower than use deepcache. Is this supposed to be ? How to save more memory?
TensorRT supports dynami…
-
### Search before asking
- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and found no similar bug report.
### Ultralytics YOLO Component
Expo…
-
When I was running the benchmark for Llama 70b, I found that all of the activation values are zero.
'''
python build.py
--model_dir /code/tensorrt_llm/models/Llama-2-70b-chat-hf/
--dtype float16…
-
## Description
Fresh install of `pip install tensorrt==10.2.0`
Following engine build crashes on Ubuntu 22.04.4 LTS:
```
from polygraphy.backend.trt import EngineFromNetwork
EngineFromNet…
-
when I try to use torch2trt in yolactedge evaluation:
```
python eval.py --trained_model=../Downloads/yolact_edge_54_800000.pth --score_threshold=0.8 --top_k=100 --image=test_Color.jpg --use_tensorr…
-
* Ubuntu 22.04
* Gcc 11.4.0
* torch 2.2.1+cu121
* python3.9
when I run
```bash
python waymo_preprocess.py --data_root data/waymo/raw/ --target_dir data/waymo/processed --split trai…
-
## Description
When performing Resnet18 PTQ using TRT-modelopt, I encountered the following issue when compiling the model with TRT.
First off, I started with a pretrained resnet18 from torchvi…