camel-ai / crab

CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents. https://crab.camel-ai.org/
https://crab.camel-ai.org/
194 stars 28 forks source link

[Spike] Concurrent Evaluation Solution #51

Open WHALEEYE opened 3 weeks ago

WHALEEYE commented 3 weeks ago

Description

Explore solutions for running the tasks concurrently.

Notes

WindowsAgentArena supports concurrent task running. We can look into this. https://github.com/microsoft/WindowsAgentArena

Expected Output

A report of the possible solution, including how to run the task concurrently, along with any drawbacks found that can be improved. A detailed report could help a lot in the future development.