-
Hi~ First, Your implemention is awesome, thank you! But I have some question.
1.What's the meaning of 'mm' in function 'contextual_attention'
2.I'am confused that after 'xi' and 'wi_normed' conv, th…
-
### 🚀 The feature, motivation and pitch
I am unable to find the clean implementation of local multi-headed self-attention in pytorch geometric. I found three types of multi-head attention, one Transf…
-
https://arxiv.org/abs/2103.02143
-
flash attention是否支持RTX8000
-
关于第九章这一部分:
为了简化数据处理,这里我们并没有将 [CLS]、[SEP]、[PAD] 等特殊 token 对应的标签设为 -100,而是维持原始的 0 值,然后在计算损失时借助 Attention Mask 来排除填充位置。
attention mask对于cls的位置是1。“active_loss = attention_mask.view(-1) == 1”会包括cls。是否需…
wli33 updated
8 months ago
-
`The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_…
-
Hi.
The code runs fine.
But I am learning Vit and want to check the attention map.
Where should I modify it? First of all, the current code doesn't seem to output attention weights.
-
**Steps to reproduce:**
1. Open https://www.qax-camp.com.ua/certificate/1273226CB-1276EB9D4-7B5A0/ - Certificate
2. Pay attention to the distance between the body and the footer.
**Actual re…
-
should get a big improvement from working from pixels rather than vqgan
-
Line 66 In the encoder_decoder function throws an error, TypeError: list indices must be integers or slices, not tuple.
Let me know if you need more details. Thanks.