Closed nicola-corbellini closed 1 year ago
I added the support for some of the arguments that the server accepts, but that was arbitrary according to what I though to be useful. Should add or maybe remove them all? @pieroit @Pingdred
Looks good to me! Thanks will try it out with pleasure
This PR adds two docker compose files to run models supported by the Llama-cpp server with the Cheshire Cat.
There is one version to run the model on the CPU and another that extends the previous to support GPU off-loading.