search

lucidrains / nGPT-pytorch

Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI
MIT License
201 stars 10 forks source link

What is the difference between nGPT.py and nTransformers.py #6

Closed ghgmail closed 4 days ago

ghgmail commented 4 days ago

What is the difference between nGPT.py and nTransformers.py？ And are they from two different papers? Thank you very much

lucidrains commented 4 days ago

@ghgmail nTransformer is just nGPT without the embedding and logit projection, a stack of attention and feedforwards