search

nus-apr / auto-code-rover

A project structure aware autonomous software engineer aiming for autonomous program improvement. Resolved 30.67% tasks (pass@1) in SWE-bench lite and 38.40% tasks (pass@1) in SWE-bench verified with each task costs less than $0.7.
Other
2.72k stars 288 forks source link

Question on how to obtain different repo versions #37

Closed xyliugo closed 6 months ago

xyliugo commented 6 months ago

Thank you for developing and maintaining this inspiring project!

I'm using harness/run_setup.py to obtain different versions of a repository (e.g., Django) for testing but noticed the clone_repo function in harness/utils.py doesn't switch to specific branches/tags. This results in always getting the latest version of the codebase. Is there a way to clone a repo's specific versions (e.g., tags) using the current setup, or did I miss something?

I am looking forward to your help and thanks again!

yuntongzhang commented 6 months ago

Thank you for your interest in AutoCodeRover.

I think you are referring to the SWE-bench mode. The setup_one_repo_version function in run_setup.py currently clones a repo, but does not checkout to the specific version. The version checkout is done in ACR when running on the actual task instance. (ACR reads from the tasks_map.json file to get the commit hash to checkout to, for each individual task instance. The tasks_map.json file is produced by the run_setup.py script.)

For each repo-version group (e.g. django v1.9), there is one repo cloned, but there can be many task instances in this repo-version group. These task instances are usually on different commit hashes. So instead of checking out to a specific version during setup time, the current workflow is to only do checkout when running on the individual task instances.

Hope this helps. Feel free to follow up if you have more questions.