search

ASSERT-KTH / drr

Tool & data on the correctness of Defects4 patches generated by program repair tools http://arxiv.org/pdf/1909.13694
http://arxiv.org/pdf/1909.13694
Creative Commons Attribution Share Alike 4.0 International
10 stars 6 forks source link

We must pay attention to execute correctly defects4j tests #13

Closed tdurieux closed 6 years ago

tdurieux commented 6 years ago

In the previous execution, we had A LOT of trouble to have the correct behavior from the tests. The tests are highly dependent of the environment. Some test that should not fail fail. It was a little bit better on grid5k.

We should pay attention to this in our evaluation

martinezmatias commented 6 years ago

Hi Thomas Yes, I agree. We can create an script that executes the fixed version from defects 4j (the manual patched)and checks all tests pass, otherwise it reports the bugs ids that have 1 or more failing

On Fri, May 25, 2018, 3:46 PM Thomas Durieux notifications@github.com wrote:

In the previous execution, we had A LOT of trouble to have the correct behavior from the tests. The tests are highly dependent of the environment. Some test that should not fail fail. It was a little bit better on grid5k.

We should pay attention to this in our evaluation

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kth-tcs/defects4-repair-reloaded/issues/13, or mute the thread https://github.com/notifications/unsubscribe-auth/AHcXnjP-i1yHca6KAX3yKJXmswoxh5qbks5t2AtIgaJpZM4UOAZY .