Open ye opened 7 years ago
Hmm, will look into these later but at the moment I'm inclined to trust mine, since I believe I'm very slightly more accurate on c++, and javascript comments should be the same as c++. I'm hoping to put together a script soon to identify the files with the largest discrepancies for manual testing. Will get back to you when I do.
Re timing: was that cold cache on loc and warm on cloc? Can you try running loc twice? I just got 160x faster testing them both against a large code base (openbsd). If not, let me know.
@cgag re: discrepancies yes, I think if you can have a set of baseline comparison tests that would be great and it would be very helpful to find bugs too since cloc
is battlefield tested and proven for the most part.
re: speed improvement stats. I think you are right, unfortunately I didn't time the first run but subsequent runs were much faster (~0.25s to ~0.28s)!!! Now I wonder what were the stuff that got cached? And where does the cached data get stored?
For the timing, the operating system caches files it accesses in memory by default, so the first time you read it, it has to get it from disk, but the second time you try to read a file, it should be read from memory, which is much much faster. The OS will use any free memory to cache files until an application needs it.
If so, I wonder why cloc
is timed at the same ball park (~1 min) on repeated runs?
That's because cloc
is CPU bound. It counts more slowly than it reads off disk, so the CPU is the bottleneck, which caching files in memory doesn't help. For loc
, reading off of disk is the slowest portion, so making it faster through caching provides a huge speed up.
I ran this on the valgrind repository (checkout r16117).
cloc/loc reported different results cloc:
5146 text files.
4437 unique files.
7445 files ignored.
http://cloc.sourceforge.net v 1.60 T=11.85 s (243.7 files/s, 126250.3 lines/s)
--------------------------------------------------------------------------------
Language files blank comment code
--------------------------------------------------------------------------------
C 1185 97930 107742 635880
Expect 921 24146 6947 451965
C/C++ Header 324 13612 24090 54026
XML 136 3870 733 21655
Assembly 57 2573 3271 8428
make 79 983 470 7989
C++ 28 1376 1377 7138
Teamcenter def 13 0 213 4658
m4 1 531 4 3913
Perl 17 729 518 3290
Bourne Shell 107 538 634 2160
XSLT 6 189 125 1152
Bourne Again Shell 8 95 131 377
Haskell 4 109 70 250
XSD 1 17 10 211
Korn Shell 1 31 24 150
CSS 1 10 4 53
--------------------------------------------------------------------------------
SUM: 2889 146739 146363 1203295
--------------------------------------------------------------------------------
loc:
--------------------------------------------------------------------------------
Language Files Lines Blank Comment Code
--------------------------------------------------------------------------------
C 1192 839672 97549 106976 635147
C/C++ Header 324 91728 13613 24072 54043
XML 136 26258 3870 732 21656
Makefile 83 9991 1071 505 8415
Plain Text 49 10557 2593 0 7964
Assembly 57 14272 2573 4524 7175
C++ 28 9891 1377 1377 7137
Autoconf 14 5751 800 1361 3590
Perl 4 2694 419 129 2146
Bourne Shell 3 621 14 4 603
Haskell 4 429 109 70 250
CSS 1 67 10 4 53
--------------------------------------------------------------------------------
Total 1895 1011931 123998 139754 748179
--------------------------------------------------------------------------------
I've run a file-by-file diff (only on .c files) and here's the result. Lines prefixed with ">" are from loc, "<" are from cloc. A lot of off-by-1 or 2, more interestingly there are certain files where loc reports 0 lines.
I can't say that cloc is 100% correct but on the few files I inspected loc indeed miscounted the number of lines.
Also an example of something loc would report wrongly: (C file)
Bool h_clo_partial_loads_ok = True; /* user visible */
/* Bool h_clo_lossage_check = False; */ /* dev flag only */
loc would return 2. cloc returns 1. Edit: fix for this
Edit 2: On this file it fails because it's ISO_8859_1 encoded and the code assumes every file is utf8
Good catch on the whitespace, I guess I should have bothered with it. I looked at the PR and it looks correct but chars.nth(pos) is probably less than ideal since nth is O(n) and we should be safe to just index into trimmed due to the is_character_boundary checks at the top of the loop.
Thanks for the valgrind diff. I'll test out the whitespace change and start digging into any differences. In my experiences cloc is off by one fairly often, but any larger differences should be interesting.
Are you going to add support for other forms of encoding, or is the tool going to be limited to utf8 files only?
Chiming in on this, I have a repo where both cloc and loc vastly misses the number of lines for Python code (60k for cloc, 8.8k for loc, 93k in reality (!)). Let me know if you want some data regarding this @cgag !
Edit: tokei seems to get the numbers 100% correct
I just cloned and cargo built
loc
from the repo. Here are the results I've tested on the exact same test code base.As you can see there are quite significant discrepancies reported by the two programs. If
loc
were a re-implementation ofcloc
then I would expect the discrepancies to be small if there were any.PS:
loc
took around 2-3 seconds to finish, it would be nice to have the elapsed time reported in the result output as well. Andcloc
took almost a minute, so it's about 20-30x improvement not the 100x as claimed.