antimatter15 / alpaca.cpp

Locally run an Instruction-Tuned Chat-Style LLM
MIT License
10.25k stars 906 forks source link

can not start interactive mode in Windows #70

Open liulirun opened 1 year ago

liulirun commented 1 year ago

I am following the README for windows, compile the code. when trying to start chat.exe , the interactive chat never appears. my windows version is win10, x64 CPU the powershell shows below:

`PS C:\study\AI\alpaca.cpp> .\Release\chat.exe main: seed = 1679245184 llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ... llama_model_load: ggml ctx size = 6065.34 MB llama_model_load: memory_size = 2048.00 MB, n_mem = 65536 llama_model_load: loading model part 1/1 from 'ggml-alpaca-7b-q4.bin' llama_model_load: .................................... done llama_model_load: model size = 4017.27 MB / num tensors = 291

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | VSX = 0 | main: interactive mode on. sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000

PS C:\study\AI\alpaca.cpp>`

kawashirov commented 1 year ago

Have the same issue. Whatever launch options I use it just exits with no errors after printing sampling parameters.

DartPower commented 1 year ago

i too have this issue

W1Real commented 1 year ago

Same problem, it gets stuck using 100% of CPU of roughly a single core. And does nothing afterwards, I just force closed it.

dhirajsuvarna commented 1 year ago

For me also exe exits without throwing any error -

Below is the output.

F:\projects\gpt\alpaca\alpaca-win>chat.exe
main: seed = 1679412237
llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ...
llama_model_load: ggml ctx size = 6065.34 MB

F:\projects\gpt\alpaca\alpaca-win>
W1Real commented 1 year ago

Btw, I fixed mine by just compiling in my machine instead of using the executable from the zip release.

Just like how the README.md tells.

cmake .
cmake --build . --config Release

Once you compile, it will appear in the builds folder.

dhirajsuvarna commented 1 year ago

@W1Real - thanks this solved my problem too

alextomskras commented 1 year ago

for xeon E5-1650v3 need disable AVX2 and recompile))))