search

SpoonLabs / nopol

Automatic program repair system for Java based on dynamic analysis and code synthesis with SMT. Also contains the code of Dynamoth.
https://hal.archives-ouvertes.fr/hal-01285008/document
GNU General Public License v2.0
97 stars 40 forks source link

Nopol generates an overfitted patches with a test suite, but a correct patch with a subset of the test suite #210

Closed Amirfarhad-Nilizadeh closed 3 years ago

Amirfarhad-Nilizadeh commented 3 years ago

I observed that Nopol generates an overfitted patches for a buggy program with a test suite. However, when I used a subset of the test suite, Nopol generates a correct patch.

What can be the reason for this behavior?

Thank you in advance for your help and time.

monperrus commented 3 years ago

This happens.

This is because too much tests make an SMT spec which can be only satisfied by hard-wiring corner cases in the patch, which is consequently overfitting.

See @zhongxingyu's paper Alleviating Patch Overfitting with Automatic Test Generation: A Study of Feasibility and Effectiveness for the Nopol Repair System (doi:10.1007/s10664-018-9619-4)