AndraxDev / speak-gpt

Your personal voice assistant based on OpenAI ChatGPT.
https://play.google.com/store/apps/details?id=org.teslasoft.assistant
Apache License 2.0
290 stars 59 forks source link

request: a more advanced model management #53

Closed thiswillbeyourgithub closed 6 months ago

thiswillbeyourgithub commented 11 months ago

Hi,

Thanks a lot for the latest release!

I find it a bit strange that you app allows changing the backend to something else than openai but does not make it easy to use different types of models frequently. You are incentivized to set the system message and chose a model and forget about it.

I think it would be great if :

  1. the assistant popup allowed for choosing the model from a predefined list
  2. a new menu item existed to manage that list. It would consist of a tuple "backend" + "model name" + "system message" + "temperature" + "advanced settings".

Those "advanced settings" could be anything supported by the backend. Things like logit bias, top_p etc. This is not that big of a deal for openai but can be very useful if using LocalAI and other models IMO.

With the recent AI news, I think we're headed for a multiplication of different chats for different use. You can need a quick and cheap chatgpt, a slow expensive gpt4, use your own finetuned model as a RAG etc. I doubt that anyone is using the finetuned feature of openai with SpeakGPT if they are forced to only use it.

I think it boils down to if you want your app to be used rather for "power users" or for regular joes. But I also think it could be a killer feature to allow such modularity because otherwise you'll end up in the very competitive real of AI assistants. I just think it would be great if I could, when calling the assistant, choose which assistant I want to talk too!

thiswillbeyourgithub commented 7 months ago

I thought about it some more and I think the best first step would be to add a dropdown to the chat assistant to allow choosing a system prompt. That would pair nicely with your Prompt store.

This way I could have for example a pre defined prompt "translate to japanese" and another "explain in one sentence the meaning of this".

That would greatly enhance the versatility of your app.

AndraxDev commented 6 months ago

Starting from SpeakGPT I added quick settings so you can quickly change model, system message and other params

thiswillbeyourgithub commented 6 months ago

Hi, thank you very much it sounds awesome but I can't find anything about it in the UI. Is there a new button somewhere? I promise I looked...

AndraxDev commented 6 months ago

Open a chat and click it's name. In the next release I will add API endpoint per chat and logit biases so don't create new issues touching this topic.

thiswillbeyourgithub commented 6 months ago

Awesome that looks incredible and stunning! Thank you so much