-
Hello
get docker image 0.6.0. Just tried to run the two demo command:
1. docker run -it --rm --gpus all \
-v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.6.0 \
bash -c "cd /project && \
…
-
**Description**
CUDA Graph not work in tensorrt backend. The model config as below:
```
platform: "tensorrt_plan"
version_policy: { latest: { num_versions: 2}}
parameters { key: "execution_mode"…
-
Run the below cm commands for several times and always failed at the same place:
(cm) tomcat@tomcat-Dove-Product:~$ cm run script --tags=run-mlperf,inference,_find-performance,_full,_r4.1-dev \
…
-
### Checklist
- [X] I have searched related issues but cannot get the expected help.
- [ ] 2. I have read the [FAQ documentation](https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/faq.md) but …
-
### System Info
I have searched the repo here and the main server repo but don't see any information on either a) support for Safetensors (many models are saved that way on HF) and also b) whether th…
-
Hello, I have a suggestion for a notebook -- an **example of a cuml trained model being exported so it can be served by TensorRT.**
More information on TensorRT:
- https://docs.nvidia.com/deeplear…
-
```dockerfile
#Base Image
FROM nvcr.io/nvidia/tritonserver:24.04-trtllm-python-py3
USER root
RUN apt update && apt install --no-install-recommends rapidjson-dev python-is-python3 git-lfs curl uuid…
-
Checklist
- [x] I've prepended issue tag with type of change: [bug]
- [ ] (If applicable) I've attached the script to reproduce the bug
- [ ] (If applicable) I've documented below the DLC image/doc…
-
### Describe the issue
In a scenario where multiple GPU devices are available, when selecting the TensorrtExecutionProvider and choosing device_id = 0, the model infers perfectly. However, when usi…
dat58 updated
2 months ago
-
Whenever I am running MLperf Inferencing for Llama2-70b in a docker container, I am getting this below error. I deleted the container image and run again but still same error.
Host server is running …