search

thunlp / UltraChat

Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)
MIT License
2.22k stars 114 forks source link

Question about the model size #21

Open Tebmer opened 1 year ago

Tebmer commented 1 year ago

Thanks for your awesome work and open your model weights!

I have one small question that why the size of 13B model weights in Huggingface https://huggingface.co/openbmb/UltraLM-13b/tree/main is roughly 53GB? (The original 13B LLAMA model is around 26GB) :)

yulinchen99 commented 1 year ago

Hi, thanks for your interest in our work. The reason is that we saved the delta weights in torch.float32, so it is almost twice as large.