-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A…
-
In response to another issue, I started writing up some thoughts on how what I'd do if I were designing this crate's API from scratch, but realized it would probably be a better fit as a standalone di…
-
### 🚀 The feature, motivation and pitch
# Problem
I am currently working with structured outputs and experimented a little with VLLM + Outlines. Since our JSON Schemas can get quite complex the …
-
### System Info
python==3.10.15
cuda==11.8-8.8.1
torch==2.4.0
The latest version of code
GPU A100_80G * 8
### Who can help?
@ziyuwan @Ge
### Information
- [X] The official example scripts
- […
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of…
-
repro code (add it to `tests/basic/main.rs`):
```rust
fn integration_path(path: &str) -> String {
let dir = env!("CARGO_MANIFEST_DIR");
format!("{}/tests/integration/data/{}", dir, path)…
-
2024-08-01 11:12:51 - Your app is available at http://localhost:8000
2024-08-01 11:12:53 - Translated markdown file for zh-CN not found. Defaulting to chainlit.md.
You are using the latest version o…
-
## Description
- Replace the old `msgpack` swift with a new solution.
- Fix crashes if the data is not a `msgpck`
- Better performance
-
I am using a pretrained model from
https://huggingface.co/speechbrain/asr-transformer-transformerlm-librispeech
with `EncoderDecoderASR` to decode the `test-clean` and `test-other` datasets
fro…
-
### 🚀 The feature, motivation and pitch
vLLM provides a guided generation via outlines, e.g.
```Python
from outlines import models, generate
model = models.VLLM(llm)
generator = generate.choice(…