[feature] support param to limit workers

boyter / scc

Sloc, Cloc and Code: scc is a very fast accurate code counter with complexity calculations and COCOMO estimates written in pure Go

MIT License

6.8k stars 267 forks source link

[feature] support param to limit workers #513

Closed cemiboou closed 1 month ago

cemiboou commented 2 months ago

Hi,

Is there a flag to limit the number of job workers?
ConfigureLimits can support by flag?

for example :
my test

time GOMAXPROCS=1 ~/opensource/scc/scc --by-file --no-cocomo --no-complexity --format json --output ./res.json

GOMAXPROCS=1 ~/opensource/scc/scc --by-file --no-cocomo --no-complexity json 3.96s user 7.59s system 30% cpu 37.680 total

time GOMAXPROCS=16 ~/opensource/scc/scc --by-file --no-cocomo --no-complexity --format json --output ./res.json

GOMAXPROCS=16 ~/opensource/scc/scc --by-file --no-cocomo --no-complexity jso 3.76s user 7.16s system 91% cpu 11.933 total

boyter commented 2 months ago

Currently no there is not. Its based on the amount of CPU's the system reports.

Seems like something that could be useful to add in though.

As you have noticed you can control the number of of threads using GOMAXPROCS but I can see it being useful to tweak each one.

boyter commented 1 month ago

$scc -h

      --file-list-queue-size int           the size of the queue of files found and ready to be read into memory (default 8)
      --file-process-job-workers int       number of goroutine workers that process files collecting stats (default 8)
      --file-summary-job-queue-size int    the size of the queue used to hold processed file statistics before formatting (default 8)
      --directory-walker-job-workers int   controls the maximum number of workers which will walk the directory tree (default 8)

This is now sitting in master. You can now control each of the sizes however you wish as you can see above.