./kim_herzig/test_data.md

[timm]

After review, relabel to 'reviewTwo'. After second review, relabel to 'EditorsComment'.

[gousiosg]

Review template

Before filling in this review, please read our Advice to Reviewers.

(If you have confidential comments about this chapter, please email them to one of the book editors.)

Title of chapter

Gotcha: Testing less without sacrificing code quality?

URL to the chapter

https://github.com/ds4se/chapters/blob/master/kim_herzig/test_data.md

Message?

What is the chapter's clear and approachable take away message?

If we use historical data about the bug finding effectiveness of tests, we can cut down testing costs significantly.

Accessible?

Is the chapters written for a generalist audience (no excessive use of technical terminology) with a minimum of diagrams and references? How can it be made more accessible to generalist?

Yes

Size?

Is the chapter the right length? Should anything missing be added? Can anything superfluous be removed (e.g. by deleting some section that does not work so well or by using less jargon, less formulae, lees diagrams, less references).? What are the aspects of the chapter that authors SHOULD change?

I think it is mostly fine as is. I 'd make it sharper by removing the introductory paragrarph (too generic and too verbose in my opinion) and just starting with par 2

Gotta Mantra?

We encouraged (but did not require) the chapter title to be a mantra or something cute/catchy, i.e., some slogan reflecting best practice for data science for SE? If you have suggestion for a better title, please put them here.

Almost; I'd use something along the lines of the core message, e.g., "Learn from your tests' run history to cut down costs"

Best Points

What are the best points of the chapter that the authors should NOT change?

The descriptions of the two testing metrics; short, to the point and pointing the reader to the appropriate publication in case he/she needs more information.

[minkull]

Review template

Title of chapter

Gotcha: Testing less without sacrificing code quality?

URL to the chapter

https://github.com/ds4se/chapters/blob/master/kim_herzig/test_data.md

Message?

What is the chapter's clear and approachable take away message? written in one line or less

Assessing test cases based on the history of test results can help improving test efficiency and effectiveness.

Accessible?

Is the chapters written for a generalist audience (no excessive use of technical terminology) with a minimum of diagrams and references?

Yes

How can it be made more accessible to generalist?

This chapter has many very interesting ideas, which I believe will be beneficial to both practitioners and researchers. However, the flow of the sentences expressing these ideas could be improved. For example, the section entitled

“The impact of short release cycles”

currently starts with the following sentence

“Due to its weight especially for large products, anything we can do to increase the effectiveness and efficiency of test execution has immediate effect on product development.“

Instead, this section could first say that the short release cycles result in short time to run tests. This leads to the need for improving the efficiency and effectiveness of testing, which in turn leads to the question “which tests to run?”, which is the title of the next section.

??

Size?

Is the chapter the right length?

Yes

Should anything missing be added?

The chapter could briefly discuss the experience of the author with selecting test cases based on test history. Has test history been used in practice before? How beneficial was it?

Can anything superfluous be removed (e.g. by deleting some section that does not work so well or by using less jargon, less formulae, lees diagrams, less references).?

No

What are the aspects of the chapter that authors SHOULD change?

The flow of the sentences could be improved and the chapter would benefit from a couple of proof-readings to fix English language issues. See some suggestions in the end of this review.

??

Gotta Mantra?

We encouraged (but did not require) the chapter title to be a mantra or something cute/catchy, i.e., some slogan reflecting best practice for data science for SE? If you have suggestion for a better title, please put them here.

The current title is ok.

??

Best Points

What are the best points of the chapter that the authors should NOT change?

The topic and insights / ideas given in the chapter are very good.

??

Suggestions on English issues

I may not be the best person to give suggestions on English language as I am not a native speaker! However, you may wish to consider the suggestions below:

In the introduction:

Change

“The number of devices and configurations modern software systems are supposed to handle is massive, and the number is still rising every year.”

to

“The number of devices and configurations modern software systems are supposed to handle is massive, and is rising every year.”

The following sentence is a bit long and difficult to read. I recommend re-writing it.

“Additionally, modern software is supposed to be omnipresent, leading to systems that can be used as on premise products, for example: traditional Office installation on a PC, and as services: Office 365 in your browser.”

Change

“However, achieving both, high complexity and short release cycles is a challenge…”

to

“However, achieving both high complexity and short release cycles is a challenge…”

In section “The impact of short release cycles”

Change

“The time spend on verification”

to

“The time spent on verification”

The use of “:” is incorrect in the following sentence:

“We need to think of testing as a risk management tool: select the right tests for the right changes and we need to ensure that the test we are executing are highly effective and highly reliable.“

“Testing more than functional correctness” in the following is unfinished:

“Some tests might be more effective than others might. However, deciding the effectiveness and reliability of tests and when to execute them is not trivial. Testing more than functional correctness Often, testing is associated with checking for functional correctness.“

[kimherzig]

Thanks for all the great feedback. I will revise and improve the chapter accordingly.

[timm]

would a better title: "A Little Less Testing: or How to Avoid Wasting Time on Unproductive Tests"

Also, i challenge you to consider the overall flow, how to sharpen and bring the main message to the audience, sooner.

[kimherzig]

Addressed comments and accepted challenge. Hope the message gets clear much sooner. Also simplified sentences. The title changed too: "There's never enough time for all the testing we want". This fits the outline better and is already part of the message. Chapter in the beginning shorter.

[timm]

@kimherzig anyway to make (a very small) throw to
http://dl.acm.org/citation.cfm?id=2635910

e.g. at least, end of para1: Here at Microsoft (as well as other large software organizations [3]) we have learned that the effort associated with testing must be carefully monitored and managed.

also, totally optional, but if you are feeling kind towards that work, maybe add just a sentence or two on their findings?

[kimherzig]

Will do this ...

[timm]

@kimherzig :+1: good to go!