kyegomez / LongNet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
https://discord.gg/qUtxnK2NMf
Apache License 2.0
663 stars 63 forks source link

Import issues #12

Closed jtrechot closed 11 months ago

jtrechot commented 11 months ago

Hello,

Thank you for the great effort to provide a version of LongNet! 😃 I noticed that I have some import issues, for example in the LongNet/model.py file:

ModuleNotFoundError: No module named 'LongNet.transformer'

I think this is due to some modification of the imports that used to be from LongNet.Transformer import LongNet but was changed to from LongNet.transformer import LongNet

Would it be possible to have a check on that ? 🔧

Thank you!

Upvote & Fund

Fund with Polar

kyegomez commented 11 months ago

@jtrechot Hey please excuse me for the issue, I've fixed it now. We're still working on the model architecture to get it ready for training and democratization.