researchart / rose7re20

3 stars 1 forks source link

Reviews on submission 35 #14

Open fabianodalp opened 4 years ago

fabianodalp commented 4 years ago

The assigned reviewers are going to post their reviews on this submission within this issue. The same thread will be used also to support the interaction with the authors.

Reviewers, please check STATUS.md to determine which badges the artifact is applying for. A description of the badges can be found here: https://re20.org/index.php/artifacts/. You will also receive an e-mail with further instructions shortly.

ltpangmao commented 4 years ago

The authors have applied for the badges of Reusable and Available.

1) Available The coding data is available on Zenodo, and the corresponding link is clearly presented.

2) Reusable

The current description of the available data is too brief to understand and reuse the data. Instead, the potential users of this data will probably have to find the corresponding paper and read through the entire 12 pages to grab the details they need.

I would suggest the authors follow the criteria of the “Reusable” badge (https://re20.org/index.php/artifacts/) and enrich their data description to make it more self-contained. At least the following issues should be considered: • Explicitly associate the data with the RE paper (since it has been accepted), and updated corresponding links once the paper is officially published. • Briefly summarize the research objectives and studies, clarifying the context under which the data is produced. • It would be nice to briefly introduce the coding technique, at least providing a reference to the grounded theory approach you used. • The meaning of each column in the codebook should be further explained, as the names of the columns are not very informative. If such things have been clearly presented in the paper, the authors could briefly summarize them and make an explicit connection to corresponding sections/paragraphs in the paper.

fabianodalp commented 4 years ago

@pfv610 I now assigned you officially to this artifact and you can check it

colinwerner commented 4 years ago

@ltpangmao thank you for the comments regarding our artifact submission. I have updated our submission based on your comments, here is a summary of what I have done: 1.) I have added the conference information to the Zenodo page, as per your recommendation. I will update it further once the paper is officially published (I have created a reminder for August 31 to remind me :)). 2.) Summarized our research, including the context of where our data came from (both here and on Zenodo). 3.) Added more information on our coding technique (both here and Zenodo) 4.) I enhanced the column descriptions (both here and Zenodo)

Please let me know if I am missing anything else!

ltpangmao commented 4 years ago

@colinwerner The revised version looks much better! Now I am fine with both the available badge and the revisable badge.

fabianodalp commented 4 years ago

I clearly assigned the "available" badge. We will finalize on the "reusable" badge depending on the feedback from the second reviewer, which is due today.

fabianodalp commented 4 years ago

We (@MonaRahimi and I) have checked the artifact and we agree with awarding the reusable badge. So, this review process is completed too!