search

OpenAutoCoder / Agentless

Agentless🐱: an agentless approach to automatically solve software development problems
MIT License
745 stars 88 forks source link

Can Agentless Execute the generated code? #19

Open MuhammadIkrmaSharaf opened 3 months ago

MuhammadIkrmaSharaf commented 3 months ago

I would like to know: 1) If, once a fault is localized and the agent enters the repair phase and generates code patches, it can execute the code? 2) Additionally, if multiple patches are created, what mechanism is used to select the best patch, and on what basis are these patches distinguished? 3) Finally, does it commit the code changes to the files?

brutalsavage commented 3 months ago

Sorry for the delayed reply,

If, once a fault is localized and the agent enters the repair phase and generates code patches, it can execute the code?

Currently we do not have any implemention that directly execute any arbitary code (We do run previous regression tests as a filtering steps)

Additionally, if multiple patches are created, what mechanism is used to select the best patch, and on what basis are these patches distinguished?

We do generate multiple patches per bug (this is something that be configured). For the selection mechanism we first filter out any patches that do not pass the previous regression tests, apply normalization, and then perform majority voting to select the final patch. Please see the paper for more detailed description.

Finally, does it commit the code changes to the files?

Currently, all patches we generate are in the git diff format. Therefore, one can simply run git apply directly on our generated patch to apply and then commit the code changes.

flyingdutchpig commented 2 months ago

Sorry for the delayed reply,

If, once a fault is localized and the agent enters the repair phase and generates code patches, it can execute the code?

Currently we do not have any implemention that directly execute any arbitary code (We do run previous regression tests as a filtering steps)

Additionally, if multiple patches are created, what mechanism is used to select the best patch, and on what basis are these patches distinguished?

We do generate multiple patches per bug (this is something that be configured). For the selection mechanism we first filter out any patches that do not pass the previous regression tests, apply normalization, and then perform majority voting to select the final patch. Please see the paper for more detailed description.

Finally, does it commit the code changes to the files?

Currently, all patches we generate are in the git diff format. Therefore, one can simply run git apply directly on our generated patch to apply and then commit the code changes.

I could not find the code doing "regression test", could you please point out its location? Thanks