Doriandarko / maestro

A framework for Claude Opus to intelligently orchestrate subagents.
3.49k stars 569 forks source link

Feature Request: Support for Google or Bing Search API, and/or a Web Scraping Tool, and/or Implementing an Internal Rate-Limit for Models #64

Open Eipckz opened 4 days ago

Eipckz commented 4 days ago

Maestro's capabilities with search are really great in my opinion, Tavily is a great AI Search tool, but I think if support was added for Google Programmable Search Engine or Bing Search API (which are also free, but Google's PSE is the best imo) Maestro can be significantly enhanced by implementing in-depth access to external information sources rather than searching about a topic and receiving a summary, when the search is ported into Maestro, the Orchestrator/Agents can have access to ALL of the information.

Furthermore, equipping Maestro with web scraping capabilities would unlock access to more targeted information. Users could directly request data from specific websites, or Maestro could autonomously initiate scraping when a subtask requires specific details.

While leveraging external APIs like Gemini 1.5 Pro offers significant advantages, Maestro has a hard time when trying to use Gemini 1.5 Pro with it's very low API requests (currently 2 requests per minute on the Free tier), I was wondering if there was a way for the user to be asked if they would like to submit their rate limits for their APIs so that Maestro can stay below those limits? I simply tried using Gemini 1.5 Pro as just the Orchestrator Agent, and every time, Maestro would fail to complete the task after running a couple agents by saying that I was reaching a usage limit, referring to the 2 requests per minute of 1.5 Pro. Now, the obvious fix to this is just using 1.5 Flash instead or really any model with higher limits (I had no issues using Claude, and I never used OpenAI or Ollama or local models, but would assume they are fine as well). An ideal mechanism in the code could track API call frequency and implement strategies such as introducing pauses, queuing requests, or dynamically switching to alternative models or APIs when approaching usage thresholds.

If any of these ideas are already in the works of being implemented into Maestro, could you please let me know? Thanks for developing an amazing tool (which helped to create this request by the way).