search

khoj-ai / khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (e.g gpt, claude, gemini, llama, qwen, mistral).
https://khoj.dev
GNU Affero General Public License v3.0
16.18k stars 794 forks source link

[IDEA] using function calling to integrate more tools #841

Open thinker007 opened 4 months ago

thinker007 commented 4 months ago

Describe the feature you'd like

give agent ability add other tools rather than build search and web browser

for example integrade dify workflow and tools

sabaimran commented 2 months ago

hey @thinker007 , could you describe more of what you're looking for? Do you want another tool similar to Dify? https://dify.ai/blog/dify-ai-workflow

thinker007 commented 2 months ago

hey @thinker007 , could you describe more of what you're looking for? Do you want another tool similar to Dify? https://dify.ai/blog/dify-ai-workflow

yes dify

DavidLMS commented 2 weeks ago

I think it would be beneficial to allow agents to define custom actions in a more generic manner, similar to OpenAI's approach with function calling in GPTs. This would offer greater flexibility for integrating various workflows and tools, extending beyond the built-in options currently available.

thinker007 commented 2 weeks ago

I think it would be beneficial to allow agents to define custom actions in a more generic manner, similar to OpenAI's approach with function calling in GPTs. This would offer greater flexibility for integrating various workflows and tools, extending beyond the built-in options currently available.

or let devs integrate tools made from dify , we don't need second dify, what we need is integrate dify tools workflow into khoj agents

thinker007 commented 2 weeks ago

By the way dify can turn tools into rest https api, if khoj can let devs integrate general http service it will be great!

felipemeres commented 4 days ago

I agree, even a simple implementation of a tool framework that lets users setup different request formats to api endpoints and call them with assigned /commands would be great.

sabaimran commented 4 days ago

These sound like really great ideas! Can you give me an idea of some tools you'd integrate? What would they do, how would you use them, how would you handle the responses? Be as detailed as possible, as that would help us design the feature.

I think the ideal tool might be something that can be called via an http request, like @thinker007 mentioned, where of course you can define the input parameters / body structure, and how to handle the response.

Unless the tool can just be text in - text out from an LLM.

I think custom code execution would probably be out of scope.

thinker007 commented 4 days ago

These sound like really great ideas! Can you give me an idea of some tools you'd integrate? What would they do, how would you use them, how would you handle the responses? Be as detailed as possible, as that would help us design the feature.

I think the ideal tool might be something that can be called via an http request, like @thinker007 mentioned, where of course you can define the input parameters / body structure, and how to handle the response.

Unless the tool can just be text in - text out from an LLM.

I think custom code execution would probably be out of scope.

Yes out there is plenty of agent tools just like dify, in khoj we just need a good way to integrate them

felipemeres commented 4 days ago

The External Knowledge feature of Dify could be a good reference. Essentially you setup the api endpoint and any agent can call this function to retrieve its output as context.