State-of-the-art Keywords JIT in comparison with Keywords + Release Level

[HuyTu7]

[HuyTu7]

Prec, Pd, Pf, F1, Precision_all, F1_all : relative delta, positive and higher are better IFA : absolute delta, negative and lower are better Bold cells are median values of deltas

Results Analysis: 1/When looking on all releases, there is no distinctions between the 3 keyword, fastread, and human buggy when building defect prediction model 2/Incremental learning provides similar results as statte-of-the-art all previous/past available releases [:i_version] 3/state of the art JIT is using JIT data collection to train for JIT defect prediction. However, the results show that, by just studying release level is very effective to better predict than the state of the art JIT defect prediction (which maybe only for computational projects)

• win on most metrics (except PCI20 since it not relevant to defect prediction power)
• high recall and high false alarm
• less data (1.2, 1.5, 5, 10 times smaller)
• less computation of metrics (only 1 time per release instead of abinit: 155, libmesh: 422, lammps:186, mdanalysis:142 commits median)
• new set of metrics of the understanding tool is more granulated that help the dataset built on release level to predict JIT bug maintenance level than JIT data collection.

Conclusions:

• incremental learning is enough (no need to track all of the history)
• release is good enough to learn from instead of doing JIT to understand predict future maintenance of the project - learning from less data but cohesive understanding of the whole project
• human-in-the-loop cannot provide better model in JIT bugs maintenance model. Possibly, the high false alarm bugs collected from keyword model combining with better granulated features from understand tool enough to understand the JIT bugs maintenance. So no improvement or difference.