-
:red_circle: **Title** : Image Captioning with Attention Mechanism
:red_circle: **Aim** : Develop an image captioning system using an attention-based model.
:red_circle: **Brief Explanation** : ā¦
-
Hi, I was testing fused attention fp8 in the tutorial on L20 GPUļ¼the test code is as following:
```python
@pytest.mark.parametrize("Z, H, N_CTX, HEAD_DIM", [(1, 2, 1024, 64)])
@pytest.mark.parameā¦
-
### Bug description
It seems that they updated the Gemma v1 2B weights. Something to look into:
```
ā” main ~/litgpt litgpt chat checkpoints/google/gemma-2b
{'access_token': None,
'checkpoint_ā¦
rasbt updated
1 month ago
-
Hi,
Llama 3 trains like this
> We trained the models on sequences of 8,192 tokens, using a mask to ensure self-attention does not cross document boundaries.
I see you have something like thisā¦
-
After the recent update Regional prompt dissapeared from UI.
UPDATE:
I found the error in logs.
```
[-] ADetailer initialized. version: 24.8.0, num models: 28
*** Error loading script: attentā¦
-
### Question
Hi, thanks for such a great work!
I would like to ask you how many images have you used for your attention diversity studies? (in Fig 4)
It would help me a lot if I get a response ā¦
-
Hello, thanks for sharing the work, it is very inspiring. I wonder if you can share the attention extraction and visualization script used for creating Figure 2 in the paper?
-
### Bug description
When using `litgpt generate` on models with softcapping, `build_mask_cache` creates mask as `torch.bool`
https://github.com/Lightning-AI/litgpt/blob/ef9647cfa7cd73e03b0e29126bfeā¦
twaka updated
1 month ago
-
### System Info
python 3.10
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [X] My own modified scripts
### Tasks
- [ ] An officially supportā¦
-
Dear AnyDesign Authors,
Thank you for your fantastic work! And I want to ask some questions:
1. The QćKćV in Fashion-Guidance Attention (FGA) Module confuse me, I want to know the reason for thisā¦