Consider techniques from official training paper

[EwoutH]

Microsoft released a new paper, which contains details and tips on training a ternary LLM. Might be useful!

• https://github.com/microsoft/unilm/blob/master/bitnet/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[kyegomez]

@EwoutH Yes, i have integrated the new codes!

[github-actions[bot]]

Stale issue message