marcom / Llama.jl

Julia interface to llama.cpp, a C/C++ library for running language models
MIT License
25 stars 2 forks source link

Fixed inference on Metal GPU backend + updated docs for run_* programs #5

Closed svilupp closed 9 months ago

svilupp commented 9 months ago

Default parameter changes:

Added README reference how to start the server.

svilupp commented 9 months ago

As part of this, I've noticed that run_chat is infinite :D CTRL+c control sequence gets eaten by REPL, so we can't kill it... Will open an issue

svilupp commented 9 months ago

This one is good to go.

I'll fix the SIGINT issue separately, but we first need to merge this.