search

src-d / hercules

Gaining advanced insights from Git repository history.
Other
2.63k stars 334 forks source link

Commit classification #188

Open vmarkovtsev opened 5 years ago

vmarkovtsev commented 5 years ago

Add Hercules analysis to classify commits. There must be a "bugfix" class. Other classes are a subject of discussion.

Available features:

Relates to https://github.com/src-d/poc-pluralsight/issues/6#issuecomment-457708560

TODO for @Jan21: add here the list of relevant papers

m09 commented 5 years ago

For the potential classes, the ones I have in mind would be:

Then an interesting point to discuss is: do we want a single class per commit or one class per line/block or multiple classes per commit (without line/block tagging).

EgorBu commented 5 years ago

About classification - it's possible to collect some labels from PRs in huge repositories like

Pattern mining

I like an idea of pattern mining. I would suggest making commit deduplication + community detection / (or clustering / topic modeling) instead of classification. Why:

deduplication + community detection

The way how to do it the same as in apollo:

So after it will be possible to query and receive communities and their descriptions / labels.

topic modeling

How to - standart topic modeling pipeline:

So after it will be possible to query and receive topics for commits.

One more point - that the first step (feature extraction) could be reused for several different approaches.

vmarkovtsev commented 5 years ago

@EgorBu Can you please transplant this comment (great job btw!) to #187 This is the issue for commit classification.

Jan21 commented 5 years ago

Thanks a lot! @m09 I've read few papers about commit classification and what people did in the past was to classify to 5 categories:

Here are links to the papers which are using this taxonomy (to be precise, the last three use only the first 3 categories):

@EgorBu I was also thinking that I will first try to do something unsupervised like topic modeling. From what I have read it seems that the feature engineering is most important here. For example, what people use instead of raw diffs is some kind of taxonomy of code changes (class added, statement deleted, etc.) which should be described in this paper: Classifying Change Types for Qualifying Change Couplings. In one paper they also reported that even the info about the author has some predictive power. I will now look at what kind of data would be easy to get for every commit.

m09 commented 5 years ago

@Jan21 interesting taxonomy. It maps quite well with the intuitive categories we come up with. I have to say I still like the last two though (Feature addition and Non functional) because if we try to fit a feature addition in the first 3 it should probably end up in Adaptive, but it doesn't feel right, and if we try to fit documentation or CI in Perfective it's already a bit better but not great either.

r0mainK commented 5 years ago

So I'll be taking over this issue at some point - I expect in a month or two. I read the literature in the past couple days, and thought about it. Regarding taxonomy, I think using the one these guys established might be a good idea, as it separates perhaps more accurately then academics do the different kind of commits - and also conveys more information. The types would thus be:

With this taxonomy, I think that we will mostly need to work on the first 4 types of commits, as the others can be inferred relatively easily using file names and types, or simple parsing in the case of the style commits. I am not sure about separating the refactor and performance commits - it makes sens but will probably be hard to do. Also not sure the revert types are really useful.

Thoughts @m09 ?

m09 commented 5 years ago

I also find your taxonomy easier to interpret for end users. If we want to go beyond unsupervised learning, we need to find a way to get data (preferably automatically), likely from GitHub issues as @EgorBu mentioned in a previous comment. It might prove hard/impossible to get data for this precise taxonomy though.