-
### Your current environment
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.6 LTS (x86_64)
GCC ve…
-
Hello, I would like to know how to perform inference with TinyLLaVA-3.1B? Simply replacing the `model_id` in the script of tiny-llava-v1-hf with TinyLLaVA-3.1B results in an error: 'You are using a mo…
-
Hi,
Context:
- I'm experimenting with both `Ellma` and `gptel` for the moment, as I find nice stuff in both.
- Until recently, both `gptel` ans `Ellama` where working correctly
- Sometimes I ne…
-
### Your current environment
```text
Python version: 3.12.3
PyTorch version: 2.3.1+cu121
```
### How you are installing vllm
```
pip install vllm
```
Building wheels for collected packages:…
-
### What happened?
symbol not found compile error for Mac metal build. If I wind back a week with "git reset --hard master@{"7 days ago"}" it builds and executes fine.
2023 M2 MBP
### Name and Ve…
-
### Your current environment
vllm(0.3.3) on ray(2.10.0) cluster deployed by docker on 2 nodes with 2 GPU(Tesla T4) each.
linux environment
root@ai151:/vllm-workspace# env
NV_LIBCUB…
-
### Model description
Hi all,
currently, the microsoft/Phi-3-mini-128k-instruct is not supported by text-generation-inference. As displayed in the following error:
```
2024-04-25T12:45:45.28…
-
In section 4.1, `We train the projector and LM backbone, and freeze the vision encoder at both stages.`
Do you train the LLM in the first stage?
-
Hi, I observed that the results in Fig. 7(C) were obtained from training with the LLaVA dataset using the base recipe. However, these results are notably higher than those reported in this paper (http…
-
Great work! It seems that the code didn't include the qwen llm backbone part?