search

semperai / amica

Amica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.
https://heyamica.com
MIT License
581 stars 92 forks source link

Maybe n_keep is missing for llama.cpp #73

Open leucome opened 6 months ago

leucome commented 6 months ago

I felt like the chat bot was losing memory when I went over the context size. I done a search of n_keep in amica project files to specify the amount of initial token I want llama.cpp to keep. Then I could not find any mention of n_keep in the code.

I would like to make sure it keep all the initial prompt. I dont want the chat bot to lose track of who it is the middle of a conversation.

Until then I will try changing llama.cpp server default.

kasumi-1 commented 6 months ago

Hi!

Yes, I think it would be nice if all of these sorts of options could be specified (maybe in advanced setting) for different backends.

leucome commented 6 months ago

Yes, I think it would be nice if all of these sorts of options could be specified (maybe in advanced setting) for different backends.

While setting up a character, I though that it would be convenient if the backends/advanced setting could also be attached to the character cards.

slowsynapse commented 4 months ago

Arbius has a $200 AIUS bounty for this issue!

Brief: Add advanced settings to include n_keep , context size, and also ability to save these in a user built list (e.g. for a particular model)

Please read carefully:

To begin work on a bounty, reply by saying “I claim this bounty” - you will have 48 hours to submit your PR before someone else may attempt to claim this bounty.

To complete the bounty, within 48 hours of claiming, reply with a link to your PR referencing this issue and an Ethereum address. You must comply with reviewers comments and have the PR merged to receive the bounty reward. Please be sure to focus on quality submissions to minimize the amount of time reviewers must take.