-
### 🚀 The feature, motivation and pitch
Recently, we read a paper where the vLLM team proposed a method called **SmartSpec**.
We believe that the research, which dynamically adjusts the speculation …
-
### Your current environment
docker: v0.5.0post1
model: DeepSeek-Coder-V2-Lite-Instruct, DeepSeek-V2-Lite-Chat/
which version support these models???
### How would you like to use vllm
…
-
### Feature Name
Lepton AI
### Feature Description
Research about Lepton AI
### Research Findings
# Lepton AI
Lepton AI is a cutting-edge provider of AI infrastructure and services, designed t…
-
### Your current environment
The output of `python collect_env.py`
```text
python collect_env.py
Collecting environment information...
2024-09-23 17:57:46.577274: I tensorflow/core/util/po…
-
This document includes the features in vLLM's roadmap for Q2 2024. Please feel free to discuss and contribute to the specific features at related RFC/Issues/PRs and add anything else you'd like to tal…
-
pip install vllm (0.6.3) will force a reinstallation of the CPU version torch and replace cuda torch on windows. pip install vllm（0.6.3）将强制重新安装CPU版本的torch并在Windows上替换cuda torch。
> >
> >
> > I don…
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
WARNING 10-27 01:12:13 rocm.py:13] `fork` method is not supported by ROCm. VLLM…
-
### Your current environment
The output of `python collect_env.py`
```text
$ python3 collect_env.py
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
…
-
### Your current environment
The output of `python collect_env.py`.
```text
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyT…
-
### Your current environment
```text
$ python collect_env.py
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used …