-
Hi Flash-Attention Team, Are there any plans to support Attention Sink style (https://arxiv.org/pdf/2309.17453v1.pdf) attention maps for causal language models? TIA!
-
We are working on annotation documentation for MF-to-BP relations and would like to assess the extent to which relations, other than 'part of', have been used to link MFs to BPs in Noctua.
We would…
-
### Author Pages
https://aclanthology.org/people/c/chen-zhang/
### Type of Author Metadata Correction
- [X] The author page wrongly conflates different people with the same name.
- [ ] This author …
-
The following only covers reorganization, not outright trimming
**PART I**
- [x] Move all the shap and related discussion to model exploration
- [x] Move model list in LM chapter to last chapter,…
-
I run a LLaVA system as presented in this repository in a docker compose setup using official Cuda docker images and run into an error on some systems with my custom trained models.
On a server using…
-
Dear ComfyUI team,
I hope this email finds you well. My name is richard, and I am one of a developer of Hunyuan DIT, an innovative and effective model that utilizes the DIT architecture. Our projec…
-
I tried converting Google Gemma 2B models to TfLite. Found it ending in failure
### 1. System information
- Ubuntu 22.04
- TensorFlow installation (installed with keras-nlp) :
- TensorFlow l…
-
Can HyperAttention be used with a key_padding_mask to prevent padding tokens from being attended to in bidirectional attention? I understand this doesn't matter in the causal case, but is important fo…
-
The input generation, inference, and embeddings/logits extraction functions (as appropriate) `tfsemb_main.py` should be moved into separate scripts for `causal`, `mlm`, and `seq2seq` models.
-
Hey,
Great to see LISA implemented here.
As for the background, I am trying to finetune models with LORA other techniques on domain data but the Task i am doing is Causal LM is Next word Predict…