search

JHubi1 / ollama-app

A modern and easy-to-use client for Ollama
Apache License 2.0
597 stars 45 forks source link

The behavior system overwrites model definition. #29

Closed nojoyy closed 4 months ago

nojoyy commented 4 months ago

I recently began messing around with model files to make dedicated custom AI chatbots using Ollama. It works great and was entertaining. I then tried to use that same model with the android client, and found that it doesn't work at all.

I imagine you are overwriting the SYSTEM flag via the API call. I'll probably fork and look into it, but the reason I made a Issue is because I then noticed that the Chat Descriptions for previous chats DO act like the character AI... This is a little unexpected for how I understood the API to work.

For demonstration, here is the relevant chat log from my conversations with a custom Snoop Dogg. AI.

Conversation did not act like expected Screenshot_20240630-125800

But conversation titles are definitely influenced Screenshot_20240630-125824

JHubi1 commented 4 months ago

Yes, that's quite easy to explain. The chat uses the chat message API. Its system message overwrites the one from the model file. Titles use the completion API, and there no system message is set. I'll look into a fix, shouldn't be too difficult. Maybe I'll add a toggle to disable adding system prompts, so the one from the model file is used.

JHubi1 commented 4 months ago

Could you try this build? ollama.zip Does it solve the issue? (you have to disable the usage of system prompt in behavior settings)

nojoyy commented 4 months ago

Works great, tysm.

nojoyy commented 4 months ago

For anyone having issues, refresh the model in the chat after toggling system behavior. Also make sure the model is actually working on host machine, not all models seem to work with model-files. Obviously if you aren't loading in massive amounts of data into the ai, the system message serves as a solid substitution for playing around.