./zimmermann/no-one-size-fits-all.md

[timm]

After review, relabel to 'reviewTwo'. After second review, relabel to 'EditorsComment'.

[minkull]

Review template

Title of chapter

No "One Size Fits All"

URL to the chapter

https://github.com/ds4se/chapters/blob/master/zimmermann/no-one-size-fits-all.md

Message?

What is the chapter's clear and approachable take away message? written in one line or less

A general model that predicts defects well for all projects is unlikely to exist…

Additional message: … and this is not a problem if there is enough data to create a model for a specific project! For projects with few or no data, not all hope is lost — some recent research have been obtaining encouraging results.

Accessible?

Is the chapters written for a generalist audience (no excessive use of technical terminology) with a minimum of diagrams and references?

Yes

How can it be made more accessible to generalist?

In order to alert the reader of the potential problem of not having enough within-project data, the last paragraph of the chapter could start with some sentence like:

“But… what if we don’t have enough data from within a project in order to create a predictive model?”

??

Size?

Is the chapter the right length?

Yes

Should anything missing be added?

The question “Why” asked in the second paragraph of the chapter could be repeated again later on when its answer is summarised, in order to make the link between the question and the answer clear.

Can anything superfluous be removed (e.g. by deleting some section that does not work so well or by using less jargon, less formulae, lees diagrams, less references).?

I suggest removing / changing the statement “Academics like the search for general models”, as it is too general :-) Not all academics may like the search for general models? Perhaps the sentence could be re-written as something like “There is much academic research trying to find general models”.

What are the aspects of the chapter that authors SHOULD change?

??

Gotta Mantra?

We encouraged (but did not require) the chapter title to be a mantra or something cute/catchy, i.e., some slogan reflecting best practice for data science for SE? If you have suggestion for a better title, please put them here.

The current title is very nice!

??

Best Points

What are the best points of the chapter that the authors should NOT change?

• The message being passed.
• The writing style, which is very informal and comfortable to read. In particular, I like the suspense of “One day back in 2009, … Why?”

??

Typos

“You’re not interested if your technique also works for other team” —> “You’re not interested if your technique also works for another team.

“The model Firefox model” —> “The Firefox model”

“the models are predictive” —> it may not be entirely clear to the readers what is meant by the models being predictive. It may be worth re-wording the sentence to say something like “predict defects well” or “successful in predicting defects”, even though what “well” or “successful” means is undefined until later on.

[maleknaz]

Title of chapter

No “One Size Fits All”

URL to the chapter

https://github.com/ds4se/chapters/blob/master/zimmermann/no-one-size-fits-all.md

Message?

The defect prediction model that works for one project, would not work for all the others.

Accessible?

Yes it is accessible.

As this essay has important message, it might be nice to somehow generalize it above defect prediction models and also refer to other predictive models in SE (such as effort prediction).

Size?

The chapter is in the right length.

The third paragraph could be polished a bit further. The author said “… the experiment was successful …” it is not very clear what does success means here. In other words, what was the objective of the experiment with 12 more projects? Did you expect that the predictive models work across projects or not? Lack of explanation might cause a bit of confusion specifically because at the end of the second paragraph you asked the question ‘why’ so I expected to see the answer to ‘why’ on the third paragraph.

Also in the third paragraph I would like to see the same information pattern as we had in the second paragraph. So, would be great if you can summarize the similarity between 12 projects. Were they all internet browsers? How they were related to each other?

My understanding from the last paragraph is that, one project with low quality data can benefit from predictive models based on other projects data although it might not work perfectly fine. Probably you can change the last part as “… Which can help projects with no or low quality data to ALSO benefit from predictive models” or explain a bit further.

Some typos and edits could be corrected as well:

• Fourth paragraph: “…works for other team” probably “works for other teams” or “works for another team”.
• Fourth paragraph: “If it doesn’t even better because it could …” could benefit from punctuation marks something like: “if it doesn’t, even better. It could be …”
• Fifth paragraph: “the model Firefox model will …” should be “the Firefox model will …”.
• Fifth paragraph: When referring to Barbara’s essay you meant in this book?

Gotta Mantra?

The current title is clear crystal but I have one suggestion: as the chapter is looking into the defect prediction models only, might be good to add something to the title to refer into this fact? Alternatively, some overview from other SE domains could be added INTO THE CAHPTER to further generalize the message and make the title and content fit much better together.

Best Points

The chapter has very important message that should not be missed in academic research nor in industrial domain. The essay is simple and tangible and perfectly accessible.

[tzimmermsr]

Thanks everyone for the great feedback!

I changed the title to "One Size Does Not Fit All", which reads smoother. (@timm)

@minkull

• I added the "But what if we don't have enough data from within a project in order to create a predictive model?" to the last paragraph
• I replaced the sentence with academics as suggested ("There is much academic research into general models.")
• I dropped the "Why?" question because the answer why IE does not predict Firefox doesn't fit very well into the chapter (and frankly we can only speculate). I added a transition to the third paragraph "To find out when models can be transferred from one project to another,..."
• Fixed typos

@maleknaz

• I dropped the "Why?" question because the answer why IE does not predict Firefox doesn't fit very well. Instead I added a transition to the third paragraph "To find out when models can be transferred from one project to another,..."
• I made a few additional tweaks to second/third paragraph to polish them. I don't want to add too many details from the experiment because they are not relevant for the main message.
• I agree that for an academic paper, I would need an accurate title or have to do something to further generalize the message, but this is a book targeted at a wider non-academic audience. :-)
• Fixed typos

[timm]

Thanks @tzimmermsr good to go