Closed minkull closed 4 years ago
Hi Authors,
The repository is overall well structured and contains the data utilised in the attached paper. The data is clean and simple. One downside is that is appears a large portion of the links in the URL
column are no longer accessible. See below for my notes including which links appear to be broken. However, I don't feel this creates any issue for assigning this submission a badge. Links, no matter to what resource, may eventually become unavailable. If possible, the authors should correct these links in the dataset and update the version in FigShare. With these comments in mind, I find it clear that this repository is at least "Reusable."
The authors have uploaded the data to FigShare. This resource means that the data will be available for an extended period of time for all interested parties to use. With this included, I find it clear that this repository is at least "Available".
ReadMe
Checking CSV for Quality
All fields appear to be clean: no missing or strangely formatted values
All https://conferences.computer.org/
links appear to be broken. Now sure if it is possible to replace those links with working ones.
All dl.acm.org
links appear to be working
All https://www.computer.org/
links appear to be working
Dataset Availability: Posted on FigShare License: Creative Commons 4.0 (CC4.0)
The file is easy to access and hosted on FigShare so "Available". The columns are reasonably easy to follow although it is a bit hard to see how the authors were capturing "agreement" - is it if at least one match occurred in the values selected? a final column that recorded "agreed/not agreed" and final resolution by the third arbiter would help to see if other raters were able to achieve the same high kappa score.
The main problem with the artifact is the curious choice to use URL instead of DOI. Many files are no longer accessible at the URL listed. DOI is in fact exactly for the purpose you are using URL for here - a permanent, (mostly) unchanging locator. Is it possible to replace this with DOI (i.e. you have a separate column somewhere ... ) - I'm guessing no, which diminishes the value of the artifact considerably.
dear @ovislabmonash
is @arifn the human-level GH contact for this one?
if you add a DOI then this will be "available". please see Zenodo for easy addition of DOIs
Thank you @timm and anonymous reviewers. Below I report how we addressed the reviewers' comments.
All https://conferences.computer.org/ links appear to be broken. Now sure if it is possible to replace those links with working ones.
The main problem with the artifact is the curious choice to use URL instead of DOI. Many files are no longer accessible at the URL listed. DOI is in fact exactly for the purpose you are using URL for here - a permanent, (mostly) unchanging locator. Is it possible to replace this with DOI (i.e. you have a separate column somewhere ... ) - I'm guessing no, which diminishes the value of the artifact considerably.
Thank you for noticing the broken URL. We have corrected them to the publication links in the ACM website.
The file is easy to access and hosted on FigShare so "Available". The columns are reasonably easy to follow although it is a bit hard to see how the authors were capturing "agreement" - is it if at least one match occurred in the values selected? a final column that recorded "agreed/not agreed" and final resolution by the third arbiter would help to see if other raters were able to achieve the same high kappa score.
Thank you for the feedback. We have added a column noting the agreement/disagreement between raters. We also added explanations in the readme file about how we captured agreements.
if you add a DOI then this will be "available". please see Zenodo for easy addition of DOIs
Thank you for the suggestion. We have created a public link with DOI in Figshare and updated the readme file accordingly. You can find the dataset here: https://doi.org/10.6084/m9.figshare.7749731.
https://github.com/researchart/rose6icse/tree/master/submissions/available/prevalenceValuesPublications
Contact information
jon.whittle[at]monash.edu
)gillian.oliver[at]monash.edu
)waqar.hussain[at]monash.edu
) -- Corresponding Authordavoud.mougouei[at]monash.edu
)harsha.perera[at]monash.edu
)rifat.shams[at]monash.edu
)arif.nurwidyantoro[at]monash.edu
)GitHub Id
https://github.com/ovislabmonash
applying for Available