Closed aef5748 closed 1 month ago
hello,
I'd be happy to explain. The config contains many options but for this one, the llm
option is important. All the listed items there llama
, hugging_face
, etc. are examples of configs for the LLM model to load/the LLM backend to use. Right now only the first one is used. That should be llama
for you but the new config allows to use the TaskProcessing tasks for the answer generation too (nc_texttotext
).
Now, to answer the real question,
/nc_app_context_chat_backend_data/
inside the docker container nc_app_context_chat_backend
. Use this command for that:
docker cp <path_to_gguf_file> nc_app_context_chat_backend:/nc_app_context_chat_backend_data/
/nc_app_context_chat_backend_data/config.yaml
) and set the new model's filename at llm->llama->model_path
docker restart nc_app_context_chat_backend
For llama, you can see all the available options here: https://api.python.langchain.com/en/latest/llms/langchain_community.llms.llamacpp.LlamaCpp.html
I'm currently using the nc_texttotext parameter in the config.yaml, so that my LocalAI instance (already setup in NC) handles the requests.
Everything works fine, but how can I change the default model used for the requests? Changing the "Default Completion Model to use" in "Admin Settings > Articifial Intelligence" in Nextcloud does not change anything, it is always the "gpt4" equivalent that seems to be used (Hermes-2-Pro-Llama-3-8B as I'm using the LocalAI AIO docker)
In the "Task: Context Chat" section, the selected item (and the only one available) is "Nextcloud Assistant Context Chat Provider"
what are the options shown in the model selection dropdown in "Admin Settings > Articifial Intelligence > OpenAI Integration"? Is it only "gpt4"? Changing this value would be enough to switch the model for normal tasks and context chat.
If you're running background job workers, you should run them with a timeout like so, at least 2 of them:
set -e; while true; do sudo -u www-data occ background-job:worker -v -t 60 "OC\TaskProcessing\SynchronousBackgroundJob"; done
I have all my downloaded models appearing in the list (+ the default models from the LocalAI AIO like gpt-4, gtp-4o, and so on..)
It finally worked after stopping / restarting the Docker Nextcloud containers.
I'm running 2 workers (as I need 1 for processing the Context Chat request + 1 more to handle the request when it's forwarded to the nc_texttotext) and now I understand the workers had no timeout so that explains why they were caching the old model... For those looking for more information about this and how to automate the workers, it's described on the page https://docs.nextcloud.com/server/latest/admin_manual/ai/overview.html#ai-overview-improve-ai-task-pickup-speed
Thank you!
Glad it worked out!
If I want to change the default model dolphin-2.2.1-mistral-7b.Q5_K_M.gguf to another model like as Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf or a custom-defined model, how should I modify it?