search

AndraxDev / speak-gpt

Your personal voice assistant based on OpenAI ChatGPT.
https://play.google.com/store/apps/details?id=org.teslasoft.assistant
Apache License 2.0
290 stars 59 forks source link

Add support for live information grabbing. #117

Closed ech0devv closed 6 months ago

ech0devv commented 6 months ago

I'd like to be able to ask GPT4o for the weather and get a response, rather than being thrown into a google search. This can also potentially be used for setting timers, checking time, sending texts, making calls etc.

AndraxDev commented 6 months ago

You are throwing into Google because function calling is active. Function calling is experimental and unstable function that can break your experience. I recommend not using it before it officially become non-experimental.

Also, please note that GPT-4o itself does not have access to the internet. If you want to use LLM with internet access I recommend using perplexity online model. It works faster. To set it up you need to have OpenRouter API provider. You can find it in the "Explore" tab.

This issue will be closed (but taken into consideration) because it touches undeclared functionality.

When function calling is enabled I give no warranty SpeakGPT will work as intended.

Thank you!

ech0devv commented 6 months ago

What I meant is you can give GPT4o access to the internet by telling it to say a certain codeword that triggers a weather response.

ech0devv commented 6 months ago

It works pretty well and would make the app more usable as a google assistant replacement

AndraxDev commented 6 months ago

I know how to give GPT internet access. This feature is called "function calling". This feature works in the next way: When enabled, your prompt will be extended with JSON object contains function names, params and description. GPT will decide which function to use and will return response extended with JSON object that contains function names along with their params amd parsed values. Then that functions will be called inside app.

But this feature is unstable because it invokes crashes and errors during generation. I need to fix it first before extending function calling capabilities. Launching Google Search has been added as a test that proves Function Calling is working normally. If you read README you will see that it's planned to add device routines so I think your issue in unnecessary for now. Thanks again!