search

joonspk-research / generative_agents

Generative Agents: Interactive Simulacra of Human Behavior
Apache License 2.0
16.18k stars 2.07k forks source link

anyone has a estimate on the cost per hour? #56

Open yubin-ai opened 1 year ago

yubin-ai commented 1 year ago

Thanks for the amazing project. Before I start running it I am trying to get a sense on the cost of running it. Thanks!

thechenv commented 1 year ago

+1

SpencerWF commented 1 year ago

I've just run a small test, 10 steps of in 'game' time, or 100 seconds of in 'game' time, it ran through some of that I believe then threw an error 'FileNotFoundError: [Errno 2] No such file or directory: '../../environment/frontend_server/storage/test2-simulation/movement/0.json'. The cost that OpenAI is showing for that usage time was $11.13. So I think it would be pretty expensive to run 1 step per 10 seconds of the day.

trunglebka commented 1 year ago

Mine just generate "they are sleeping..." and it cost $5 :smiley:

fatedier commented 1 year ago

Based on my testing, the model currently being used is text-davinci-003, which is much more expensive than gpt3.5. I suggest not running it for too long at the moment.

happyfire commented 1 year ago

I run 10 steps, and cost about $1.24 with 3 agents and text-davinci-003

mberman84 commented 1 year ago

Would be amazing to be able to plug in open source models to run this locally. OpenLLM or something similar could provide the API interface.

SpencerWF commented 1 year ago

I run 10 steps, and cost about $1.24 with 3 agents and text-davinci-003

Running with 3 agents instead of 25 was a smart move.

lxycopper commented 1 year ago

I run 10 steps, and cost about $1.31 with 3 agents and text-davinci-003

thulanxc commented 11 months ago

I've just run a small test, 10 steps of in 'game' time, or 100 seconds of in 'game' time, it ran through some of that I believe then threw an error 'FileNotFoundError: [Errno 2] No such file or directory: '../../environment/frontend_server/storage/test2-simulation/movement/0.json'. The cost that OpenAI is showing for that usage time was $11.13. So I think it would be pretty expensive to run 1 step per 10 seconds of the day.

10 steps will not cost $11.13. The first step the GPT generates plans for the day, and in the next 2000 steps or more the agents are sleeping, which costs very few. You can create a folder named movement in the simulation storage folder, and things will work.

thulanxc commented 11 months ago

I've just run a small test, 10 steps of in 'game' time, or 100 seconds of in 'game' time, it ran through some of that I believe then threw an error 'FileNotFoundError: [Errno 2] No such file or directory: '../../environment/frontend_server/storage/test2-simulation/movement/0.json'. The cost that OpenAI is showing for that usage time was $11.13. So I think it would be pretty expensive to run 1 step per 10 seconds of the day.

or did you run the 25-agent version? I only ran the 3-agent version. 3000 steps cost me 8 dollars.

wudi-7mi commented 8 months ago

Replace all the text-davinci-002 or 003 in this project with gpt-3.5-turbo-instruct, and it costs only $0.11 in 10 steps.