search

researchart / patterns

http://tiny.cc/sepat
Other
10 stars 6 forks source link

Lack of an artifact #12

Closed apanichella closed 3 years ago

apanichella commented 3 years ago

Among the invalid criticism, the document states "Lack of an artifact should not be grounds for rejection of the scientific paper. There may be valid reasons, such as industrial non-disclosure agreements".

While I can partially agree on that, I think there should be a different standard (weight) between "industry data", "open source data", and "personal data". I do agree that in industrial applications there are NDA. Further, studies with human participants may include sensitive data.

But papers that mine data from open-source projects from Github (or similar) have no "major" reasons not to include an artifact.

I would suggest that that bullet point for criticism is revised with regard to the context (industrial or not) of the paper on which such a criticism is valid or not

neilernst commented 3 years ago

This one seems to depend on one's perspective on open science. I think you are suggesting that absent an explanation like GDPR or NDAs, the paper should be rejected if it does not provide an artifact. This seems to stray beyond reviewing the artifact itself, and into how conferences and journals run themselves.

How about we move this to the Notes, and rephrase as

There may be valid reasons why a study/paper does not include an artifact, including legal agreements with collaborators such as non-disclosure agreements, or privacy concerns, for example, personally identifiable information or institutional review board approvals.

timm commented 3 years ago

@apanichella : not sure if this covers you comment but i combined your notes with @neilernst 's to make:

image

if that does not work for you, can you suggest other (brief) text?