-
If I'm not wrong, hypergraph convolution (HC) seem really like self-attention mechanism (or earlier, non-local network) but with different score function and aggregating function, right? Basically, th…
-
### 🐛 Describe the bug
```python
import torch
from torch.nn.attention.flex_attention import create_block_mask, flex_attention
torch.set_default_device("cuda")
@torch.compile(dynamic=True)
de…
-
# ❓ Questions and Help
Hi! I work on vision transformers and implemented the window attention, where the feature map will be divided into local windows and self-attention among tokens will only happe…
-
### 🐛 Describe the bug
I struggled a bit to get a repro, but I think this is in the realm of reasonable and identifies the behavior that causes my runs to diverge.
```python
import torch
impor…
-
Trying to run `app_flux.py` results in the following error:
```
$ python3.12 app_flux.py --offload --fp8
INFO:albumentations.check_version:A new version of Albumentations is available: 1.4.18 (yo…
-
### System Info
L4 GPU (AWS G6.12xl) with TensorRTLLM 0.11.0, running with Tritonbackends
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified …
-
Hello,
Thank you for your excellent work on this project!
While reviewing the code, I noticed a few discrepancies between the implementation and the manuscript's description, specifically in the…
-
### Description
I am calling `jax.nn.dot_product_attention` with the following line:
```
dpsa_cudnn = jax.nn.dot_product_attention(query, key, value, implementation='cudnn')
```
However, this t…
-
## Describe the bug
Describe the bug
OpenAI "get_embeddings" function can't be accessed at query time. The OpenAI client is initialised, but the check is returning before the client can be accesse…
-
Hi @lucidrains ,
I hope you are doing well. And thank you for yet another useful repo! :)
I was wondering if you have any plans to support the zigzag version of ring attention. It seems to distr…