search

zorazrw / agent-workflow-memory

AWM: Agent Workflow Memory
Apache License 2.0
210 stars 16 forks source link

Cost of running quickstarts on WebArena and Mind2Web using OpenAI #1

Closed nyck33 closed 1 month ago

nyck33 commented 2 months ago

This is very cool. It reminds me of Q-learning?
But wondering if you ever tried using vector databases vs. dictionaries like you now?

FInally, what are the approximate costs to run the quick starts?

I remember PentestGPT being over 100 dollars per run or something like that...

zorazrw commented 1 month ago

Hi! Thank you for your interest! No. Could you elaborate a bit on how vector db/dict could be used in our experiments? We didn't have an exact estimation of that, but using gpt-4 models for the entire WebArena/Mind2Web dataset probably costs hundreds of dollars, as the benchmarks cover over 2k examples and the model needs to take multiple steps to solve each.