Commit Guru 150

[Suvodeep90]

TODO

• [ ] do the sanity checks of 1385 help here?

Expectation from results:

1. adequacy of predictors (Pd > 66, pf < 33)
2. FSS Is useful
3. Hyerparameter optimization is useful
4. it all scales
5. stable conclusion across
6. stable conclusion locally

FILES

• results spreadsheet:
  • https://docs.google.com/spreadsheets/d/1_v_oyN3-_JuJkJYC7iA53TEs1frmm9po/edit#gid=1000346812

[timm]

Methods: 'ns', 'nd', 'nf', 'entropy', 'la', 'ld', 'lt', 'ndev', 'age', 'nuc', 'exp', 'rexp', 'sexp','fix' FX: commitguru: 13 attributes

[timm]

FSS

CFS(https://github.com/ai-se/bellwether_community/blob/master/src/CFS.py) paper: https://www.cs.waikato.ac.nz/ml/publications/1997/Hall-LSmith97.pdf paper: https://www.cs.waikato.ac.nz/~mhall/HallHolmesTKDE.pdf thesis: https://www.cs.waikato.ac.nz/~mhall/thesis.pdf parameter:

temporal selection: https://arxiv.org/pdf/1803.05067.pdf

[timm]

LEARNER: logistic regression

• [ ] FFT 8, FFT 16 : might be able to avoid FSS
• [ ] Dodge

[timm]

Hyper parameter optimizer

• DE on SMOTE
• DE on LogReg
• nothing on FSS

suggestions:

• don't search M+N space, search M then N

[timm]

Success criteria

• [x] recall
• [x] false alarm
• [x] IFA: section5 : https://xin-xia.github.io/publication/icsme173.pdf
• [ ] PCI@20%: probably not
• [ ] POpt20 (can't do cause of not LOCS).
• [x] popt20 surrogate
• [ ] precision is bad (http://menzies.us/pdf/07precision.pdf)

[timm]

DATA

150 projects rows:

• chart of rows per project spited on rows
• chart of ratio of defectives

[timm]

what's the PEEKING mechanism?

None. no zzz

Just the activity in the commit

[timm]

LABELLING using keyword labelling

not yet active learning yet

[timm]

TRAIN-TEST rig

• Leave one out * 150
  • everything is inside

suggestions:

• move somethings out to a one time pre-preprocessr
• don't do all, do a. tournament small groups that grow

[timm]

RELATED WORK

has anyone used this data to get PD>66 nd PF < 33 before?

1) Predicting crashing releases of mobile applications - uses some metrics collected from commit guru along with other code related metrics. (recall- between .5 to .7, prec - ~0.2)

2) Just-In-Time Bug Prediction in Mobile Applications: The Domain Matters! - uses commit guru metrics with FSS - (58(p),25(R),34(F1))

3) Software Maintenance at Commit-Time - (90.75(P) 37.15(R) 52.72(F1))