"Accurately acknowledge what the pros and cons of your technique are versus the strawman, specifically in the context of precision/recall."
My feeling is that we did do this, so perhaps the reviewers misunderstood what we wrote. We should clarify the language around the experimental results so that this is clear. My takeaways: CheckCell achieves comparable performance to normal analysis (in terms of precision/recall) for most errors but requires dramatically less effort. We also have yet to show (see issue #48) that CheckCell finds "subtle errors", but supposing that it does, we also should discuss those results here.
dbarowy
commented
10 years ago
"Accurately acknowledge what the pros and cons of your technique are versus the strawman, specifically in the context of precision/recall."
My feeling is that we did do this, so perhaps the reviewers misunderstood what we wrote. We should clarify the language around the experimental results so that this is clear. My takeaways: CheckCell achieves comparable performance to normal analysis (in terms of precision/recall) for most errors but requires dramatically less effort. We also have yet to show (see issue #48) that CheckCell finds "subtle errors", but supposing that it does, we also should discuss those results here.