Closed mgedmin closed 4 years ago
I did an strace and I see the vim process is busy reading pathnames inside my node_modules from a pipe. And I do have let g:CommandTFileScanner = "find"
in my .vimrc.
Basically, there's nothing command-t can do about it. (Well, it could ask find not to descend into node_modules but parsing the wildcards in &wildignore to construct -name node_modules -prune -o
bits on the right place of the find
command line seems complicated.)
One thing you might consider trying @mgedmin is using the Watchman scanner. It would require you to install Watchman, but it's a tool that is designed to index file hierarchies way in excess of 10k files without skipping a beat. I'd expect it would handle the kind of workload that you describe quite easily.
Actually, it is command-t's fault after all!
$ echo 3 |sudo tee /proc/sys/vm/drop_caches > /dev/null
$ time find -type f|wc -l
242228
real 0m6,485s
user 0m0,603s
sys 0m2,858s
6 seconds to find the files.
$ time vim +CommandT
# press Esc Esc Esc ZQ while command-t is scanning
real 0m49,698s
user 0m6,366s
sys 0m19,220s
50 seconds (! not 10-20!) to process them.
Also I haven't cleaned the FS caches after the find and before vim +CommandT.
A second run is faster:
$ time vim +CommandT
# press Esc Esc Esc ZQ while command-t is scanning
real 0m1,588s
user 0m1,303s
sys 0m0,901s
I wonder why! I thought find
would already cache everything that's necessary...
Yeah, not sure why there'd by a difference like that. On macOS in a big repo I see this when using the "find" scanner:
$ time find . -type f|wc -l
441719
find . -type f 1.39s user 8.29s system 39% cpu 24.627 total
wc -l 0.04s user 0.02s system 0% cpu 24.626 total
$ time vim +CommandT
nvim +CommandT 4.64s user 8.99s system 52% cpu 25.950 total
So, roughly the same inside and out. I don't expect it makes a difference, but the actual find
command run by Command-T ends up being something like this (from here):
find -L . \
-mindepth 1 \
-maxdepth 15 \
-type f \
-print0 \
-o -name .* -prune
For some reason command-t has been very slow to do the initial scan with recent versions of vim. I press \t and I see it scanning files, then it stops for a long while at 8453 files or so, nothing happens for like 20-40 seconds, and then it unfreezes and I can use my vim again.
Forcing command-t to rescan with :CommandTFlush is fast. Quitting and re-running vim and using \t is fast. Asking the Linux kernel to drop filesystem caches with
echo 3 | sudo tee /proc/sys/vm/drop_caches
is needed to make the problem reappear.I'm not sure exactly how many files there are in my tree (I wasn't able to find any diagnostic commands in :help command-t), but some bisetion with g:CommandTMaxFiles shows that there are more than 10000 but less than 10500.
I'm on Ubuntu 19.10 and vim 8.1.2291.