Feat: --cutoff and "keep_workspace_files" options - Githubissues

Significant-Gravitas / Auto-GPT-Benchmarks

A repo built for the purpose of benchmarking the performance of agents, regardless of how they are set up and how they work.

MIT License

275 stars 76 forks source link

Feat: --cutoff and "keep_workspace_files" options #261

Closed lc0rp closed 1 year ago

lc0rp commented 1 year ago

Background

When creating, debugging or fixing tests or agents' performance on tests, it'd be helpful to have an option between the configured data.cutoff, and --nc.
Similarly, it's sometimes useful to prevent workspace destruction, when debugging failing tests.

Changes

Added --cutoff (seconds) so you can override the cutoff time from commandline
Added a "keep_workspace_files" config option to prevent workspace teardown after tests

PR Quality Checklist

[X] I have run the following commands against my code to ensure it passes our linters:
```
black . --exclude test.py
isort .
mypy .
autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports --ignore-pass-after-docstring --in-place agbenchmark
```
==> Mypy had a few notes, unrelated to this code. I'm happy to help look at them later.