search

OpenNLPLab / lightning-attention

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
MIT License
182 stars 15 forks source link

Tests fail #8

Closed catid closed 7 months ago

catid commented 7 months ago

All the tests currently fail:

FAILED ops/test_lightning2.py::test_lightning2[dtype1-2-8-2048-128-64] - asse...
FAILED ops/test_lightning2.py::test_lightning2[dtype1-3-8-2048-128-64] - asse...
FAILED ops/test_lightning2.py::test_lightning2[dtype1-6-8-913-128-64] - asser...
FAILED ops/test_lightning2.py::test_lightning2[dtype1-6-8-513-128-64] - asser...
FAILED ops/test_lightning2.py::test_lightning2[dtype1-6-8-1213-128-64] - asse...
FAILED ops/test_lightning2.py::test_lightning2[dtype1-6-8-2048-16-64] - asser...

To reproduce, I checked out the repo, set up a conda env, pip install -e . and then:

cd tests
. ./script.sh

I also wrote my own unit test and it fails:

First differing element at index (0, 1, 1, 0): linear=6.259500503540039, lightning=6.514106273651123
Doraemonzzz commented 7 months ago

Due to some numerical issues with triton itself, it is reasonable for these phenomena to occur. This kernel has been used to train transnormerllm, so feel free to use it. We will optimize the test section in the future to reduce ambiguity.

catid commented 7 months ago

Sorry disregard this was a bug in my testing script.