GZoltar / gzoltar

GZoltar - Java Library for Automatic Debugging http://www.gzoltar.com
Other
77 stars 34 forks source link

[Question] Why the Fault Localization results are different when using GZoltar 0.1.1 and 1.6.0? #55

Closed Instein98 closed 1 year ago

Instein98 commented 2 years ago

Hi dear developers:

Firstly I want to thank you for your efforts in building this tool because I noticed that lots of state-of-the-art automatic program repair tools are using it. However, I am a newbie to the GZoltar. I recently found that the fault localization result can be different when using GZoltar 0.1.1 and 1.6.0 (as shown in the Table V of the paper You Cannot Fix What You Cannot Find!).

image

I am really curious about what changes led to this difference? Since the Ochiai formular should not change, is that because the way to calculate the coverage is changed? I'm looking forward to your reply. Thanks!

jose commented 2 years ago

Thanks for your interest.

I am really curious about what changes led to this difference? Since the Ochiai formular should not change, is that because the way to calculate the coverage is changed?

Afirmative. Over time we managed to improve two important aspects of coverage collection for automatic fault localization techniques: (1) instrumentation and therefore a better / more accurate coverage data and (2) execution of test cases in isolation. Those two aspects are the main reason why different versions of GZoltar produce different results.

-- Best, Jose