-
### Your current environment
vllm = 0.5.4, 0.5.5
```
super().__init__(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 46, in __init__
self…
-
### 先决条件
- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [X] 错误在 [最新版本](https://github.com/open-com…
-
使用v0.12.0docker镜像部署，启动命令如下：
sudo docker run -d -v /home/tskj/MOD/:/home/MOD/ -e XINFERENCE_HOME=/home/MOD -p 9997:9997 --gpus all xprobe/xinference:v0.12.0 xinference-local -H 0.0.0.0 --log-level de…
-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N…
-
### System Info
- `transformers` version: 4.44.0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.10.13
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.2
- Accelerate v…
-
### System Info / 系統信息
ubuntu 22.04
Docker version 27.2.0, build 3ab4256
CUDA Version: 12.5
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
- [X] docker / docker
- [ ] pip instal…
-
### Your current environment
```text
durr@learner:~/vllm$ python collect_env.py
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTo…
-
### Your current environment
For setup, I am using the version 0.5 and the vllm_openai target as part of the Dockerfile with these arguments:
```
environment:
- NCCL_SOCKET_IFNAME=eth0
…
-
Here is the development roadmap for 2024 Q3. Contributions and feedback are welcome.
## Server API
- [ ] Add APIs for using the inference engine in a single script without launching a separate se…
-
**What problem or use case are you trying to solve?**
File editing is not perfect with our current method using SWE-Agent style actions.
**Do you have thoughts on the technical implementation?**…