Open svilupp opened 9 months ago
Disabling julia's SIGINT (this signal is sent by the shell when pressing Ctrl-C) handler seems to do the trick for me:
disable_sigint() do
run_chat(model="./models/rocket-3b-2.76bpw.gguf", prompt="Say hi!")
end
First Ctrl-C interrupts llama.cpp, second Ctrl-C then returns to the julia REPL.
This should probably be added to both run_chat
and run_llama
.
@svilupp Can you try with and without gpu on macos on current main branch? And perhaps also 1 CPU thread vs multi-threaded
Expectation: When I run
run_chat
, I'd like to terminate the interactive session with CTRL+C (as per the llama.cpp manual).Problem: When I press CTRL+C, the interrupt control sequence gets consumed by REPL and is not emitted. Ie, I cannot stop it and have to restart the REPL session
MWE
Versions