search

princeton-nlp / LLM-Shearing

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
https://arxiv.org/abs/2310.06694
MIT License
533 stars 39 forks source link

Instruction tuning dataset #57

Closed kiucho closed 6 months ago

kiucho commented 6 months ago

Thanks for sharing great research.

I'm trying to reproduce the result of instruction tuning(Sheared-LLaMA and Open-LLaMA both). Can I get dataset which were used for tuning Sheared-LLaMA-ShareGPT? or weights for Open-LLaMA-ShareGPT.

Thanks.

xiamengzhou commented 6 months ago

Hi! The weights of Sheared-LLaMA-ShareGPT is provided on HF. 1.3b, 2.7b.

Please find the datasets here in the google drive.

kiucho commented 6 months ago

Thank you for your kind reply. I thought I had checked all the details of this project, but I guess I made a mistake, thanks again.