pointer to transformer (big) model

[anirudt]

🌟 New model addition

Model description

Hi, needed a pointer on how to instantiate a Transformer-big from the original Vaswani et. al. paper (Attention Is All You Need). I could only find versions of Transformer-like architectures, so would be useful if this could also be added.

Open source status

• [x] the model implementation is available: (give details): https://research.google/pubs/pub46201/
• [ ] the model weights are available: (give details)
• [ ] who are the authors: (mention them, if possible by @gh-username)

[02shanks]

@anirudt Can I work on this issue?

[VarunGumma]

Any leads on this?