search

s-JoL / Open-Llama

The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
https://huggingface.co/s-JoL/Open-Llama-V2
MIT License
30 stars 4 forks source link

有计划放出13B及以上的模型么？ #44

Closed laoda513 closed 1 year ago

s-JoL commented 1 year ago

如果是说从头预训练的受资源限制恐怕是做不到，不过在考虑用65B或33B的基于一个已有的在中文继续预训练一部分数据。