search

SpoonLabs / astor

Automatic program repair for Java with generate-and-validate techniques :v::v:: jGenProg (2014) - jMutRepair (2016) - jKali (2016) - DeepRepair (2017) - Cardumen (2018) - 3sfix (2018)
https://hal.archives-ouvertes.fr/hal-01321615/document
GNU General Public License v2.0
207 stars 106 forks source link

Effectiveness of APR tools to repair student programs #233

Open ssk1216 opened 4 years ago

ssk1216 commented 4 years ago

Are the current APR tools effective in fixing the bugs in the programming assignments submitted by the students?
https://www.shinhwei.com/tutoring.pdf paper suggests that repair hints don't seem to be of much use to students. What are the reasons for APR being ineffective when applied in the educational domain?

monperrus commented 4 years ago

I'd say we need to be careful with the wording: "effective" is too vague and too subjective.

The core scientific facts we have is a percentage of repaired student programs for a given benchmark and repair tool.

Both in https://www.shinhwei.com/tutoring.pdf and https://arxiv.org/pdf/1905.11973.pdf (benchmark IntroClassJava), this percentage is approx. 20%.