search

cheshire-cat-ai / llama-local

MIT License
6 stars 1 forks source link

Ready-to-use compose files #2

Closed nicola-corbellini closed 1 year ago

nicola-corbellini commented 1 year ago

This PR adds two docker compose files to run models supported by the Llama-cpp server with the Cheshire Cat.

There is one version to run the model on the CPU and another that extends the previous to support GPU off-loading.

nicola-corbellini commented 1 year ago

I added the support for some of the arguments that the server accepts, but that was arbitrary according to what I though to be useful. Should add or maybe remove them all? @pieroit @Pingdred

pieroit commented 1 year ago

Looks good to me! Thanks will try it out with pleasure