search

OpenNLPLab / lightning-attention

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
MIT License
184 stars 15 forks source link

No-decay unusable #11

Closed jmercat closed 6 months ago

jmercat commented 6 months ago

There is a new no decay function. It is used here but its use is prevented there. Should that last assert statement be removed?

Doraemonzzz commented 6 months ago

Yes, I'll update this later.

Doraemonzzz commented 6 months ago

Update this.