Define details of review form

[tkuhn]

This is the default review form of the submission system: https://github.com/data-science-hub/files/blob/master/review-form/default.txt

We have to adjust this to reflect our review guidelines and journal features.

[tkuhn]

Here is a first draft: https://github.com/data-science-hub/files/blob/master/review-form/version1_draft.txt

[micheldumontier]

nice. i would just change:

Relevance and importance of research question *:

• Clear relevance
• Limited relevance
• Lack of relevance

to

Significance of research question

• High significance
• Moderate significance
• Limited significance

[tkuhn]

I chose "relevance" over "significance" because the latter is ambiguous. I am worried that reviewers misunderstand it for "statistical significance" in a statistical test.

[micheldumontier]

issues related to statistical significance should be identified in the technical quality.

maybe we need to have descriptions for people. e.g. significance: Is there a strong scientific premise for the project? Does the work address an important problem or critical barrier to progress in the field?
Is scientific knowledge, technical capability, and/or clinical practice improved? Does this work change the concepts, methods, technologies, services that drive the field?

[tkuhn]

OK, I changed to to "significance" and added a explanatory question for some of the questions. See here: https://github.com/data-science-hub/files/blob/master/review-form/version1_draft.txt

[micheldumontier]

excellent

[tkuhn]

I did some work on the review form: https://github.com/data-science-hub/files/blob/master/review-form/version1_draft.txt

For novelty and technical quality, I added questions for the different types of articles:

Novelty:

• For research papers: Does the work provide new insights or new methods of a substantial kind?
• For position papers: Does the work provide a novel and potentially disruptive view on the given topic?
• For survey papers: Does the work provide an overview that is unique in its scope or structure for the given topic?

Technical quality:

• For research papers: Are the methods adequate for the addressed problem, are they correctly and thoroughly applied, and are their results interpreted in a sound manner?
• For position papers: Is the advocated position supported by sound and thorough arguments?
• For survey papers: Is the topic covered in a comprehensive and well balanced manner, are the covered approaches accurately described and compared, and are they placed in a convincing common framework?

I also added a new criterion background.

I quite happy with the current version. I think it is ready to be employed, but it would be good to hear other opinions.

[tkuhn]

Once fixed, I will also align the reviewer guidelines to the review form: http://datasciencehub.net/guidelines.html

[micheldumontier]

love it.

m.

On Wed, Jan 25, 2017 at 3:04 AM, Tobias Kuhn notifications@github.com wrote:

Once fixed, I will also align the reviewer guidelines to the review form: http://datasciencehub.net/guidelines.html

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/data-science-hub/data-science-hub.github.io/issues/24#issuecomment-275080133, or mute the thread https://github.com/notifications/unsubscribe-auth/AA8qPLgMjSt7n7JVkt25vQmPggHnEVwHks5rVywwgaJpZM4LEdpi .

-- Michel Dumontier Distinguished Professor of Data Science Maastricht University http://dumontierlab.com

[tkuhn]

This is the first version of the review form that we are going to use: https://github.com/data-science-hub/files/blob/master/forms/review-form-v1.0.txt