-
vLLM has a cousin named [aphrodite](https://github.com/PygmalionAI/aphrodite-engine) which adds EXL2, GGUF and a bunch of other handy features to it.
The good news is that the wheels already work o…
-
### Which package is this bug report for?
discord.js
### Issue description
Getting error `Unhandled 'error' event emitted, received [object Object]` although listening to both `error` and `shardErr…
-
### Your current environment
```text
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
GCC ve…
-
### Your current environment
```text
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.10 (x86_64)
GCC version: (…
-
### Your current environment
```text
python env.py
--2024-06-16 20:27:30-- https://raw.githubusercontent.com/PygmalionAI/aphrodite-engine/main/env.py
Resolving raw.githubusercontent.com (raw.gith…
calz1 updated
4 weeks ago
-
### Your current environment
```text
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and =1.17.3 and
-
I am using a gguf model on Aphrodite Engine but the issue is that i was to have context length of 8192 ctx but i can got it to load only about 4096 context length, issue is that i'm short on vram...…
-
### Your current environment
```
Collecting environment information...
PyTorch version: 2.3.0
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 2…
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS…
-
### Your current environment
I can not find any doc about this feature.
### How would you like to use Aphrodite?
_No response_