I regularly have to open files that are gigabytes in size (logfiles), or source code that contains hundreds of thousands of lines, in which case Neovim can hang for a long time on opening the file, and every action inside the file is very very slow (moving lines takes several seconds).
Some of the features which become very slow on large files include:
gitsigns
illuminate
the various treesitter plugins (and treesitter itself). I think this is because it needs to parse the file from the very beginning in order to highlight correctly, and that can take a long time. The fallback (default) highlighting in Neovim that uses regular expressions may be slower and imprecise in general, but at least it works with more reasonable speed on large files.
various other Vim features that the existing LargeFile plugin already disables
Since Doom-nvim is focused on performance I think it'd be appropriate to have a feature to deal with large files better.
It won't be pretty, but at least the file will open and you can edit it.
If I have time I can take a look at more gradual disabling of features (e.g. maybe treesitter highlighting itself is not too bad, it is just all the tree-sitter plugins that are very slow on large files), but meanwhile this might be useful for someone already.
I also have some scripts that can measure performance using hyperfine and some simple scenarios (e.g. fabricate a "large" file with thousands of lines and measure how long it takes to use a motion to go to the end and insert some characters). These scenarios are more than just the usual "vim startup time", which although important may miss the point in some cases: I think what matters is "time to first action" (similarly to how browser latency is sometimes measured as 'time to first byte'): if my editor is frozen for minutes and won't respond to anything I type when I open a file, then the fact that the editor "started up" in 10ms or 100ms matters little, what should be measured is how soon before the editor is able to receive user input. (And then all else being equal, a faster startup "display" time is of course desirable).
The scripts aren't quite finished yet, so they are not included in this PR. You can see their current state at https://github.com/edwintorok/doom-nvim/blob/main/tools/profileit.sh and https://github.com/edwintorok/doom-nvim/blob/main/tools/compare_perf.sh.
It is a bit tricky to automate bootstrapping of neovim from 2 different commits to compare performance in a reproducible way, and I don't currently have a way to automatically check whether bootstrapping of doom succeeded or not, if it hasn't then it'll print various warnings/errors and not initialize completely which will cause any performance comparisons to be invalid.
I regularly have to open files that are gigabytes in size (logfiles), or source code that contains hundreds of thousands of lines, in which case Neovim can hang for a long time on opening the file, and every action inside the file is very very slow (moving lines takes several seconds).
Some of the features which become very slow on large files include:
Since Doom-nvim is focused on performance I think it'd be appropriate to have a feature to deal with large files better. It won't be pretty, but at least the file will open and you can edit it.
If I have time I can take a look at more gradual disabling of features (e.g. maybe treesitter highlighting itself is not too bad, it is just all the tree-sitter plugins that are very slow on large files), but meanwhile this might be useful for someone already.
I also have some scripts that can measure performance using
hyperfineand some simple scenarios (e.g. fabricate a "large" file with thousands of lines and measure how long it takes to use a motion to go to the end and insert some characters). These scenarios are more than just the usual "vim startup time", which although important may miss the point in some cases: I think what matters is "time to first action" (similarly to how browser latency is sometimes measured as 'time to first byte'): if my editor is frozen for minutes and won't respond to anything I type when I open a file, then the fact that the editor "started up" in 10ms or 100ms matters little, what should be measured is how soon before the editor is able to receive user input. (And then all else being equal, a faster startup "display" time is of course desirable).The scripts aren't quite finished yet, so they are not included in this PR. You can see their current state at https://github.com/edwintorok/doom-nvim/blob/main/tools/profileit.sh and https://github.com/edwintorok/doom-nvim/blob/main/tools/compare_perf.sh. It is a bit tricky to automate bootstrapping of neovim from 2 different commits to compare performance in a reproducible way, and I don't currently have a way to automatically check whether bootstrapping of doom succeeded or not, if it hasn't then it'll print various warnings/errors and not initialize completely which will cause any performance comparisons to be invalid.
What do you think?