-
## Description
(A clear and concise description of what the bug is.)
Model artifacts are in the (TRT-LLM) LMI model format:
` aws s3 ls ***
PRE 1/
2024-10-25 14:59:…
-
### Your current environment
Failed to import from vllm._C with ImportError("/usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /tmp/.conda/envs/vllm_env/lib/python3.10/…
-
## ❓ Question
I have a PTQ model and a QAT model trained with the official pytorch API following the quantization tutorial, and I wish to deploy them on TensorRT for inference. The model is metaforme…
-
**What is the bug?**
Model is getting stuck in deploying state while registering it on the cluster. We have seen cases where the model is not found on the few nodes.
Scenario
1. Model stuck in DE…
-
AS a user
I WANT see a dropdown list that contains the list of deployment models
SO THAT I can select a deployment model to use
-
Hi, May I ask how to deploy the model in the close-loop sim environment? Especially how to input target points into the model to guide the ego car? Looking forwards for ur reply!
-
Well. I really don't know how to deal with it because deployment was more straight forward.
I read documentation and I am not finding if there is a way to create my ansible playbook to iterate deplo…
-
/kind bug
**What steps did you take and what happened:**
I tried to deploy a MLflow model using KServe.
However, the inferenceService Pod was in an error state. I checked the logs using `kubect…
-
/kind bug
**What steps did you take and what happened:**
When trying to deploy a large model (e.g. `granite-20b-code-instruct`) the `Pod` terminates before completion.
I use a `ServingRuntime` …
-
- Copilot Chat Extension Version: 1.243.0
- VS Code Version: 1.95.1
- OS Version: Windows 11 Pro 10.0.22631 Build 22631
Steps to Reproduce:
1. Generate a code using copilot chat
2. Paste the code i…