search

Significant-Gravitas / AutoGPT

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
https://agpt.co
MIT License
166.45k stars 44.06k forks source link

Run benchmark tests from command line #5417

Closed pplonski closed 11 months ago

pplonski commented 11 months ago

Duplicates

Summary 💡

The UI for running tests is beautiful. I would love to use it at the end of development, when I'm sure that everything is working as expected. For development I would prefer a command line tool to run selected tests.

Examples 🌈

No response

Motivation 🔦

Using command line is faster than UI for development.

pplonski commented 11 months ago

I've found some docs about CLI here https://github.com/Significant-Gravitas/AutoGPT/blob/master/CLI-USAGE.md

I have no idea, how to run selected test with selected agent.

Swiftyos commented 11 months ago

Hi @pplonski The UI is the preferred method of running tests now. However, if you want to use the CLI this is the comand

./run benchmark start my_agent --test=WriteFile

You can get a list of the available test with:

./run benchmark tests list

pplonski commented 11 months ago

@Swiftyos thank you for response. The command that you provided does not work. It starts a server and do nothing ...

My output:

piotr@komp2:~/sandbox/mljar-agent/autogpts/mljar-agent$ ./run benchmark start mljar-agent --test=WriteFile
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
The currently activated Python version 3.8.10 is not supported by the project (^3.10).
Trying to find and use a compatible version. 
Using python3 (3.10.13)

       d8888          888             .d8888b.  8888888b. 88888888888 
      d88888          888            d88P  Y88b 888   Y88b    888     
     d88P888          888            888    888 888    888    888     
    d88P 888 888  888 888888 .d88b.  888        888   d88P    888     
   d88P  888 888  888 888   d88""88b 888  88888 8888888P"     888     
  d88P   888 888  888 888   888  888 888    888 888           888     
 d8888888888 Y88b 888 Y88b. Y88..88P Y88b  d88P 888           888     
d88P     888  "Y88888  "Y888 "Y88P"   "Y8888P88 888           888     

                8888888888                                            
                888                                                   
                888                                                   
                8888888  .d88b.  888d888 .d88b.   .d88b.              
                888     d88""88b 888P"  d88P"88b d8P  Y8b             
                888     888  888 888    888  888 88888888             
                888     Y88..88P 888    Y88b 888 Y8b.                 
                888      "Y88P"  888     "Y88888  "Y8888              
                                             888                      
                                        Y8b d88P                      
                                         "Y88P"                v0.1.0

[2023-10-02 11:05:54,747] [forge.sdk.agent] [INFO]  📝  Agent server starting on http://localhost:8000
pplonski commented 11 months ago

Sorry, there was something on my end. All is working fine! Thank you!

pplonski commented 11 months ago

Important notice, you need to run server in other process in order to use command line tool.