メモリ不足対策 - Githubissues

aki-ph-chem commented 3 weeks ago

リポジトリのissueによれば20GiBほどのswapがあれば8Bのモデルの変換もうまくいくらしい。

aki-ph-chem commented 3 weeks ago

Laptop(OS: Arch Linux x86_64,Kernel: 6.11.4-arch2-1,CPU: 13th Gen Intel i5-13500H (16) @ 4.700GHz,Memory: 15655MiB)に20GiBのswap file を作成してから変換処理を行ったら1bitLLM/bitnet_b1_58-3Bの変換処理及び実行に成功した。

aki-ph-chem commented 3 weeks ago

同じLaptop環境で HF1BitLLM/Llama3-8B-1.58-100B-tokensの変換処理を行ったら途中で死んだ。 swapは10GiBほど使用されていた。

aki-ph-chem commented 3 weeks ago

そのときのメッセージ

 $ poetry run python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s
INFO:root:Compiling the code using CMake.
INFO:root:Downloading model HF1BitLLM/Llama3-8B-1.58-100B-tokens from HuggingFace to models/Llama3-8B-1.58-100B-tokens...
INFO:root:Converting HF model to GGUF format...
ERROR:root:Error occurred while running command: Command '['/home/aki/run_BitNet/BitNet/.venv/bin/python', 'utils/convert-hf-to-gguf-bitnet.py', 'models/Llama3-8B-1.58-100B-tokens', '--outtype', 'f32']' died with <Signals.SIGKILL: 9>., check details in logs/convert_to_f32_gguf.log

aki-ph-chem commented 3 weeks ago

Desktop(OS: Arch Linux x86_64, Kernel: 6.11.4-arch2-1, CPU: 13th Gen Intel i7-13700 (24) @ 5.100GHz, Memory: 31851MiB) で20GiBのswap fileを作ってから変換処理を行ったら完走できた。

aki-ph-chem commented 3 weeks ago

一応変換できたモデルは動いた。

aki-ph-chem commented 3 weeks ago

ChatGPTで適当なクイズを作って答えさせてみた。以下の問題はちゃんとした答えが帰ってきた。

If today is Monday, and the day after tomorrow is Friday, what day is it today?
There are 5 birds on a tree. A hunter shoots 2 birds. How many birds are left on the tree?
Jane is taller than Emma, but shorter than Lucy. Who is the tallest?
There are 4 red balls and 3 blue balls in a basket. If you take out 2 blue balls, how many blue balls are left?
A man has 10 apples. He eats 4 and gives 3 to his friend. How many apples does he have left?

逆にこんなクイズはだめ

The sun rises in the east and sets in the west. If it’s morning and you are facing the sunrise, which direction is behind you?

aki-ph-chem / run_BitNet

メモリ不足対策 #1