-
### Describe the issue as clearly as possible:
I cannot create an outlines generator with a mistral model and pydantic schema.
### Steps/code to reproduce the bug:
```python
import outlines
from p…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
deepspeed --num_gpus 4 ../../src/train.py \
--deepspeed ../deepspeed/ds_z3_offload_config.json \
…
-
### System Info
Package Version
------------------------ ---------------
accelerate 0.30.1
aiohttp 3.9.5
aiosignal 1.3.1
annotated-…
-
### Describe the bug
When loading the mistral and Llama model on T4 GPU, I'm getting this error
raise openllm.exceptions.OpenLLMException(f'Failed to initialise vLLMEngine due to the following er…
-
I am using vllm on a ray cluster, multiple nodes and 4 gpus on each node. I am trying to load llama model with more than one gpu by setting tensor_parallel_size=2. The model won't load. It works fine …
-
### Your current environment
```text
The output of `python collect_env.py`
```
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorc…
-
### Describe the bug
Basically getting some form of this error, either ```rocBLAS error: Cannot read /opt/rocm-5.4.0/lib/rocblas/library/TensileLibrary.dat: Illegal seek``` or ```Cannot read TensileL…
-
```
~~> nix run .#textgen-nvidia -- --api --auto-launch
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
2023-12-12 23:44:37 ERROR:Could not fi…
-
### Your current environment
```text
2024-04-02 13:00:28 (5.94 MB/s) - ‘collect_env.py’ saved [24853/24853]
Collecting environment information...
/usr/local/lib/python3.10/dist-packages/transfor…
-
### Package Name
altair, ibis-framework[duckdb], leafmap[libremap], myst
### Hub URL
nature.datahub.berkeley.edu
### Course Name
ESPM 157
### Semester Details
Fall 2024
### Ins…