-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
### Your current environment
```text
The output of `python collect_env.py`
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12…
-
Add an example implementing the "Prompt Lookup Decoding" technique:
https://github.com/apoorvumang/prompt-lookup-decoding
This should be a great exercise for people looking to become familiar wi…
-
Hi, it's me again. I don't know if you still remember me. I'm the guy who reported touchscreen fault 4 months ago with Surface Laptop 2.
This time, my new Surface Laptop 3 AMD Ryzen version w…
-
An idea that has been kicking around for years, but never written down:
The current definition of `int` (and correspondingly `uint`) is that it is either 32 or 64 bits. This causes a variety of pro…
-
Some llama integration tests (e.g. `test_llamacpp_various_regexes`) fail for llama-cpp-python >= 0.2.38.
Investigate this further.
-
# Compatibility Report
- Name of the game with compatibility issues: Halo Infinite
- Steam AppID of the game: 1240440 (Believed to be this ID)
Note:
Creating a preliminary post regarding this …
-
### Your current environment
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: …
-
### Your current environment
H100 (but I believe it happens in any machine)
### 🐛 Describe the bug
```
--enable-chunked-prefill --num-max-batched-tokens 2048 --kv-cache-dtype "fp8"
```
S…
-
### Your current environment
The output of `python collect_env.py`
```text
GPU NVIDIA RTX 5880
```
### Model Input Dumps
_No response_
### 🐛 Describe the bug
I've noticed a…