search

acmsigsoft / artifact-evaluation

2 stars 4 forks source link

Artifact Extendability #4

Open jon-bell opened 4 years ago

jon-bell commented 4 years ago

To what extent should reviewers try to extend an artifact (build on a tool, analyze a dataset in a new way, or provide new inputs/configurations to that tool or analysis), in comparison to simply running it?

sbaltes commented 4 years ago

I would consider running/verifying the artifact as the baseline and everything else nice to have, but not required.