How many characters counts as a cursory code review?

andymeneely commented 10 years ago

We need to determine a threshold for the number of characters (non-quote, non-URL, etc.) that are the cutoff for a "cursory" code review. Having a paper to cite would be great. We need a reason for whatever we choose.

One option is 4 characters for just LGTM. It's the minimum. It's 17% of all code reviews.
Another option is 140 characters, which is a semi-serious joke. That's abou 40% of all code reviews.
There are many, many code reviews with ~8 characters, and if we went with 4, it's hard for us to believe that 8 is not cursory and 4 is. Especially for comments like "Otherwise, LGTM"

kaylaerdmann commented 10 years ago

Starting to look at some journals and found this: Alt Text It's auto-generated comments from a code review bot studied in this paper. Maybe we want to pick a threshold that's around the average char count of something like this?

andymeneely commented 10 years ago

Hm - I actually saw that paper presented last year in San Francisco! Not quite what we're looking for, but I'll have to think about it. Might be a nice threshold.

andymeneely commented 9 years ago

We ended up not coming up with a good threshold for a cursory review - there's no obvious cutoff from the distribution. We'll have to approach this another way.

andymeneely / chromium-history

How many characters counts as a cursory code review? #170