search

shawwn / llama-dl

High-speed download of LLaMA, Facebook's 65B parameter GPT model
GNU General Public License v3.0
4.17k stars 419 forks source link

why are the 13B, 30B and 65B models size dont match the original files? #22

Closed Coderx7 closed 1 year ago

Coderx7 commented 1 year ago

Hi, I noticed the 13B model is 12GB only. did you somehow mistakenly used the 7b model in 13b model? I also noticed the 30B and 65B models are 15GB only. the weights should look like this apparently :

model original size quantized size (4-bit)
7B 13 GB 3.9 GB
13B 24 GB 7.8 GB
30B 60 GB 19.5 GB
65B 120 GB 38.5 GB
Coderx7 commented 1 year ago

OK, this is answered here : https://github.com/shawwn/llama-dl/issues/4 basically starting from 13B, the download is multi-parts. 

"%file: tokenizer_checklist.chk"
eeec4125e9c7560836b4873b6f8e3025  tokenizer.model

"%file: 7B_checklist.chk"
6efc8dab194ab59e49cd24be5574d85e  consolidated.00.pth
7596560e011154b90eb51a1b15739763  params.json

"%file: 13B_checklist.chk"
0804c42ca65584f50234a86d71e6916a  consolidated.00.pth
016017be6040da87604f77703b92f2bc  consolidated.01.pth
0171286599e0f8858893e49fb74b71f8  params.json

"%file: 30B_checklist.chk"
f856e9d99c30855d6ead4d00cc3a5573  consolidated.00.pth
d9dbfbea61309dc1e087f5081e98331a  consolidated.01.pth
2b2bed47912ceb828c0a37aac4b99073  consolidated.02.pth
ea0405cdb5bc638fee12de614f729ebc  consolidated.03.pth
4babdbd05b8923226a9e9622492054b6  params.json

"%file: 65B_checklist.chk"
9deae67e2e7b5ccfb2c738f390c00854  consolidated.00.pth
0c4b00c30460c3818bd184ee949079ee  consolidated.01.pth
847194df776dd38f8ae9ddcede8829a1  consolidated.02.pth
3b6c8adcb5654fd36abab3206b46a0f1  consolidated.03.pth
68d61d1242597ad92616ec31b8cb6b4c  consolidated.04.pth
7f71259eaee2b906aa405d8edf39925f  consolidated.05.pth
0574e26b6891ab2cb0df7340d773fe9b  consolidated.06.pth
e5d9790df955270b836aec79462ead22  consolidated.07.pth
0306cf93a1c2c3891166fe1fd782c2b7  params.json