search

OpenAutoCoder / Agentless

Agentless🐱: an agentless approach to automatically solve software development problems
MIT License
724 stars 86 forks source link

Regression test over PASS_TO_PASS cases in SWE-bench? #38

Open Hodge931 opened 2 weeks ago

Hodge931 commented 2 weeks ago

By this line: https://github.com/OpenAutoCoder/Agentless/blob/main/agentless/test/run_tests.py#L548, I guess the regression tests are using the PASS_TO_PASS testcases in the original dataset to do the filtering?

brutalsavage commented 2 weeks ago

Hi @Hodge931 We do not use any of the PASS_TO_PASS testcases, we directly obtained all the passing tests in the original repository and then use LLM to select the list of regression tests. For more detail you can see the detailed README