-
I used the self-trained llama13b model in tensorrt-llm 0.7.1 environment, using 2 4090 gpu, running multiple batches of the dataset, generating abnormal results. However, when running a single batch i…
-
/code/tensorrt_llm# python examples/llama/convert_checkpoint.py --model_dir /code/tensorrt_llm/Mixtral-8x7B-Instruct-v0.1/ --dtype float16 --output_dir /code/tensorrt_llm/examples/Mixtral-8x7B-Instruc…
-
Hi @wondervictor ,I changed the associated config, checkpoint, and img-size in export_onnx.py.
![image](https://github.com/AILab-CVC/YOLO-World/assets/59815166/a9320cc6-19dc-469b-9136-211031244de2)
…
-
## Description
I was trying to use TRT modelopt library to quantize a resnet18 from pytorch. The code to reproduce is:
```
from torchvision import models
from torch import nn, optim
# Def…
-
### 问题确认 Search before asking
- [X] 我已经搜索过问题,但是没有找到解答。I have searched the question and found no related answer.
### 请提出你的问题 Please ask your question
solov2怎样进行模型压缩,或者tensorrt加速?
参照paddleslim里面的…
-
I have successfully generated calibrated dataset.cache file of my dataset using polygraphy. I want to load the generated calibrated cache file and create int8 engine using c++.
Function I'm using t…
-
### System Info
GPU: rtx8000
Diver version: 525.85.05
Cuda version: 12.0
Syetem: ubuntu20.04
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own mod…
-
Hi Ryan,
I am trying to create a calibration file for the ResNet-18 Caffe model. You have mentioned the below statement in another issue:
_I have created a reference for INT8 calibration on Ima…
-
### System Info
- Ubuntu
- GPU A100 / 3090 RTX
- docker nvcr.io/nvidia/tritonserver:24.02-trtllm-python-py3
- Python tensorrt-llm package (version 0.9.0.dev2024030500) installed in the docker im…
-
我按照文档要求已经按照了包含trt的paddle,
paddlepaddle_gpu-2.2.1-cp37-cp37m-linux_x86_64.whl
在设置参数--run_mode=trt_int8 --trt_calib_mode=True跑picodet导出模型的的infer.py时候报错
File "/home/vehicle_detection/PaddleDetecti…