[research] Prepare the camera-ready-version of the style-analyzer paper

[warenlg]

The style-analyzer paper located in overleaf has been accepted to MSR'19 in the Tool track. It has been reviewed by 3 academics and the comments should be addressed before submitting the camera-ready version of the paper. Deadline: March 15, 2019, 23:59 AoE

[warenlg]

List of review comments to address

General and Inescapable

• [x] The paper has to be formatted according to https://www.ieee.org/conferences/publishing/templates.html

• [x] Respect page limits (10p without references and appendices)

• [ ] List of authors

• [x] Validate the title

• [x] Removed anonymized information for double-blind purposes

• [ ] Wait for the instructions of the publisher (IEEE CPS) and submit the IEEE copyright (before completing the final paper submission)

• [x] At least one of the paper's authors must register to the conference by the camera-ready deadline, specifying the unique Paper ID

• [x] Change the attributes pruning description, and update the corresponding numbers - Vadim

Review

• [x] Typos and grammar mistakes spotted by reviewers
• [ ] Precise more clearly the input/output of the tool
• [ ] Explain the construction process of the artificial style mistakes -> these mistakes should be consistent with real style mistakes. Are those mistakes reasonable, as far as the real development is concerned?
• [ ] Explicitly define what types of code styles are considered in this paper. Indeed code style include more stuff than we pretend to fix, e.g. http://logging.apache.org/log4j/2.x/javastyle.html
• [ ] Explain the mined style in Figure 3 in more details.
• [ ] Explain the classifier in more details. Explain the feature vector: What does a vector denote? A line of code? A so called virtual node? What types of predictions can it make? Does the classifier consider the sequence of tokens?
• [x] In Page 3, what is a virtual node? What is the differences from an actual node? Do they belonog to the same kind of trees?
• [x] In Section V.A, how did we determine that our predictions are correct? Conduct an n-fold evaluation to improve the reliability of the analysis results.
• [x] How to calculate the confidence of a code style? What is the support of a code style?
• [x] Code indentation can have sematic meanings in some programming language, e.g., Python.
• [ ] In Page 1, In this paper, we solve the automatic code formatting problem at code review time,-> I do not believe that the problem is solved. The proposed approach mines only a type of code styles. There are still many open challenges in this research line.
• [x] In Page 1, Finally, the analysis performance is high enough to yield results with small time footprint - under five minutes for large pull requests -> Can the analysis time be further reduced?
• [x] In Page 2, Both NATURALIZE and CODEBUFF are mostly black boxes, and their decisions can not be straightforwardly understood by humans. -> Please explain why the two tools are black boxes, since readers can be unfamiliar with the two tools.
• [x] In Page 2, the threshold of 80 hits a good balance between keeping significant labels and cutting away outliers. We leave approximately 30%. -> Please list this number in the setting of the evaluation section.
• [ ] In Page 2, We take advantage of the sequence prediction framework to correctly model insertions and deletions -> How to model the deltas? The follow-up descriptions seem to be incorrect to model code styles. Code changes can be modelled as changes of code styles, but cannot be modelled as code styles themselves.
• [x] In Page 3, Our approach mixes sequential and structural information to draw the richest label context-> Why is it the richest? By what criteria?
• [x] In Page 3, Taking advantage of the UAST structure, we manage to compute it with logarithmic complexity. -> How to achieve this?
• [x] In Page 3, what is one-hot encoded label? What is one-hot encoded?
• [x] In Page 4, We experimentally found 0.92 to provide a good tradeoff; it keeps from 10% to 50% of the rules depending on the dataset and model. -> Please move these sentences to the evaluation section.
• [x] In page 2, when you say "[w]e leave approximately 30%." do you mean, we leave out 30%?

[m09]

Regarding authors: since this is now a tool paper and Lookout is a major part of the paper, I feel that we should include team members from @src-d/applications (if they want to be listed of course).

[bzz]

Regarding authors: since this is now a tool paper and Lookout is a major part of the paper, I feel that we should include team members from @src-d/applications (if they want to be listed of course).

If that would be possible, I think it's reasonable to try to include top#4 contributors from https://github.com/src-d/lookout/graphs/contributors which right now are Maxim, Carlos, me and David.

[EgorBu]

TBH - in case of Lookout I would love to mention much more people because it's definitely great work of all source{d}. And there are some papers with 1000s of authors like that.

[vmarkovtsev]

done