search

facebookresearch / MobileLLM

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
Other
1.16k stars 65 forks source link

Training data? #3

Closed jacqueline-he closed 4 months ago

jacqueline-he commented 4 months ago

Hi, Cool work! I'm wondering if there are any details about the data used to train MobileLLM from scratch -- I seem to have missed it in the paper. Is it similar to the data mixture used to train LLaMa-2?

liuzechun commented 4 months ago

Hello, thank you for your question. We did not use any in-house data to train MobileLLM. We are still working with our legal team on the dataset disclosure.