search

DeveloperLiberationFront / Program-Navigation-Plugin

2 stars 3 forks source link

User study does not include statistics #114

Closed jssmith1 closed 8 years ago

jssmith1 commented 8 years ago

The user test reported here is rudimentary and doesn't include any statistics. Why didn't you do a more systematic test?

jssmith1 commented 8 years ago

From rebuttal:

(3): We originally did not perform statistical tests because the sample size was very small and the evaluation was preliminary. In response to R3’s suggestion, we tested overall task completion time using unpaired two-sample t-tests and correctness using chi-squared tests. The tests failed to identify a significant difference in completion time and correctness between our tool and the baseline. We will add the results of these tests to Section VI of the paper.