-
Hi, I would like to ask why the attention mask is not used in the prefill stage.
I want to output the attention scores matrix in prefill stage. Is the code below right?
```
if spec: # s…
-
From [Algorithmic Simplicity](https://www.youtube.com/@algorithmicsimplicity):
- [x] [Why Does Diffusion Work Better than Auto-Regression? - YouTube](https://www.youtube.com/watch?v=zc5NTeJbk-k)
-…
-
### Describe the issue
I train a qat self attention model by Pytorch FX, the model can be run in libQnnCpu.so but error in libQnnHtp.so.
The model run in linux x86.
QNN: 2.20.0.240223
ERROR Messag…
-
## 一言でいうと
グラフの畳み込みを行う際にAttentionを導入したもの。各ノードを表すベクトルに対し共通重みWをかけて処理し、「ノードAにとってノードBがどれくらい重要か」を計算するために計算結果のベクトルをコンカチしてAttentionを計算する(計算は隣接ノード分のみ対象)。
![image](https://user-images.githubusercontent.co…
-
## 집현전 중급반 스터디
- 2022년 7월 10일 일요일 9시
- 김은서님 발표
- 논문 링크: https://arxiv.org/abs/1710.10903
> ### Abstract
> We present graph attention networks (GATs), novel neural network architectures that ope…
-
Venue: ICLR 2017
Summary: Proposes a new GCN architecture which utilizes an attention mechanism to learn a weight for neighbors of each node.
My opinion: This architecture is too complicated with …
-
env:
torch.__version__ = 2.0.1+cu118
onnx.__version__ = '1.16.0'
command:
python cosyvoice/bin/export_onnx.py --model_dir $dir
error logs:
/root/miniconda3/envs/cosyvoice/lib/python3.8/site-pac…
-
### 🐛 Describe the bug
When I use flex attention on one RTX 4090, I got some error.
A minimal repro:
```python
import torch
from torch.nn.attention.flex_attention import flex_attention
flex_at…
-
### Expected Behavior
Troubleshoot the flux-fp8-dev black graph
### Actual Behavior
Running flux-fp8-dev e4m3fn on 4090 using Vincennes graphs will always be black, it still appears after disabling…
-
Hi Xiaoxin,
Thanks for this comprehensive collection of works! I would like to add this interesting work into the dataset section:
- When Heterophily Meets Heterogeneity: New Graph Benchmarks and …