-
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask`…
qiufb updated
2 hours ago
-
### 🐛 Describe the bug
I'm trying to add micro-benchmark for flex attention, which is implemented by HOP. I use ```torch.utils.flop_counter.FlopCounterMode```, but it doesn't support capture FLOP f…
-
### 🐛 Describe the bug
I was trying to enable [ring attention](https://github.com/lucidrains/ring-attention-pytorch) with torch.compile, here are the issues that I encountered:
* einx triggers "un…
-
**Describe the bug**
I try to use LLaVA example and faced to key mismatch error. I am on latest commit in main branch. (094d66b)
[rank0]: RuntimeError: Error(s) in loading state_dict for LLaVAMode…
-
when running with `attention = True` the last match has the wrong values of latitude:
min and max values of the latitude for that particular batch:
```
batch 10 = 135.0 - 146.25
batch 11 = 148.5…
-
We recently released [FlexAttention](https://pytorch.org/blog/flexattention/), which automatically generates fused flashattention kernels for a diverse range of attention variants.
For example, the…
-
### Description
Perhaps I am using this function incorrectly, but I get data leaks when using `key_value_seq_lengths`. It appears as though both the `xla` and `cudnn` implementations in jax nightly…
-
Thank you for this amazing work!
I was wondering if the fp8 implementation of flash attention 3 will be able for public to use? My main concern will be accuracy (block quant may have alleviated thi…
-
Dear author, I have another question for you:
In Visual Prompt Encoder, is it stacking three layers of deformable cross-attention layer, then connecting one self attention and one FFN?
Or stacki…
-
```bash
Torch version: 2.4.0+rocm6.1
PyTorch version 2.4.0+rocm6.1 available.
Warning, cannot find cuda-compiled version of RoPE2D, using a slow pytorch version instead
Warning, cannot find cuda-c…