Enhancement idea: add /web function to search using jina

[thiswillbeyourgithub]

Hi,

Jina recently unveiled their new search tool, it allows with a simple curl to get search result from the web in a LLM friendly format.

I was thinking you might be interested in this. Here's the link : https://jina.ai/news/jina-reader-for-search-grounding-to-improve-factuality-of-llms/

That would allow web search using any LLM given, although it would require a bit of engineering to avoid reaching the max_tokens limit or being too expensive for nothing. I can suggest implementation details for that if you want.

Have a nice day!

[AndraxDev]

I think I will made such functionality paid along with speak while streaming. Remember that this app is open source and repository where you are posting your issue is only client app. I think jina search will need some backend for better performance so it is not in my plans for now. Thanks for understanding!

Also I receive no revenue from ads so I think I will replace ads with paid service. And if users will continue using my app without supporting (it not belongs to you) I will consider closing source code and making app paid.

[AndraxDev]

I tested Jina and I found that search is not working (it always return 422 http error independently on user query). Reader works but works slowly.

The solution will be user prompt > Function call > Jina Search > Embeddings contextual search > LLM process > Output.

As for now response will be too long because jina nor embeddings API have sufficient performance.

So I'm considering replacing Jina with Perplexity and make it semi-paid (users can use Perplexity without API and pay me or users can set their API key).

This solution will be better because requests will have less response time.

Algorithm: User prompt > Function calling (will automatically determine when search is necessary) > If search is necessary perplexity will generate short summary > LLM process > Output.

This algorithm will be more precise, cheaper amd faster.

If you have other ideas you can start a discussion.

[thiswillbeyourgithub]

I tested Jina and I found that search is not working (it always return 422 http error independently on user query). Reader works but works slowly.

I think they are very new and were targeted by DDOS at the time. I would be surprised if they didn't get much faster pretty soon if not already. FWIW regarding what you're saying about response time : personnaly, I am completely willing to wait 30 seconds for a request if I know it's actually doing something that complex. But yeah perplexity will always be faster.

I have other things to say about the agorithm etc but want to think some more before bothering you

[AndraxDev]

This issue has been moved into discussion https://github.com/AndraxDev/speak-gpt/discussions/125 and will be closed to not pollute issues section. Please write your new ideas about search to that discussion.