-
**Description**
I have observed quality degradation when using `formatron` for generation in comparison with no formatron. I suspect this applies to cases where both the hex representation and string…
-
### Your current environment
```
docker pull vllm/vllm-openai:latest
docker stop pixtral ; docker remove pixtral
docker run -d --restart=always \
--runtime=nvidia \
--gpus '"device=MIG…
-
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
WARNING 09-21 15:29:13 _custom_ops.py:18] Failed to import from vllm._C with Im…
-
After attempting to run the example code present on the repo I ran into the following error. Any suggestions on how to proceed? My install was through PIP. and my model file contains a locally downlo…
-
The OpenAPI specification has been updated in the repository.
Please review the changes and prepare new client version for publishing.
Release branch name: release/oas-update-commit-3cb8d3e
Pull Req…
-
The OpenAPI specification has been updated in the repository.
Please review the changes and prepare new client version for publishing.
Release branch name: release/oas-update-commit-23e14c9
Pull Req…
-
I get this error following the deepspeed-fastgen instructions:
```python
from mii import pipeline
pipe = pipeline("mistralai/Mistral-7B-v0.1")
```
The full stack trace is:
```
Loading ext…
-
Hello,
I've noticed that the prompt for title generation is not optimal. For example on my simple message `Hello`... The title I got was `💬 Hello! How can I help you today? Let me know if you have an…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the [LangGraph](https://langchain-ai.github.io/langgraph/)/LangChain documentation with the integrat…