startup time slows down gradually after last `:LuaCacheClear`

[ViRu-ThE-ViRuS]

when I clear the cache out, and relaunch neovim, startup time for me is ~3ms, but after editing some of my config files (which are cached), and reopening vim, the startup time increases (north of ~5ms)

it causes noticeable performance deficit the longer its been since the last LuaCacheClear. I have looked at the profile, and none of the files im editing (and cache invalidating) take long to load (using LuaCacheProfile), but the load time for plugins such as 'gitsigns' increases from sub 1 ms (in first few launches on clean cache), to north 2ms (after 10 mins of using vim normally, opening and closing config files, updating plugins, etc).

Please advise how to investigate this.

[lewis6991]

How are you measuring startuptime?

[ViRu-ThE-ViRuS]

im using tweekmonster/startuptime.vim plugin, :StartupTime command

[lewis6991]

Can you see if the size of the cache files is growing over time?

[clason]

I also observe this, but it's clearly correlated to the size of the cache file, so completely expected: If you lazy-load a lot of plugins, they will only be added to the chunk cache (and hence loaded at startup) when first loaded during editing.

Adding lazy-loading to impatient for a 2ms difference (which is also what I see between "naked" startup and with all plugins touched at least once) may not be worth it, though...

[ViRu-ThE-ViRuS]

yeah I mean, 2ms is not noticeable anyways but when say when ur startup time otherwise is 3ms, its a 30% increase. Furthermore, the idea of this plugin is to minimize startup time, and the small gains add up over time right.

[lewis6991]

This is just how the cache works. What could you even do to mitigate this? There is a single cache file, the bigger it gets the slower it will be to load. If we split the cache per chunk then that will lead to a lot more sys calls.

[ViRu-ThE-ViRuS]

maybe we could have multiple cached files, which are loaded on demand when plugins are lazy loaded. Is this feasible / practical?

[clason]

If we split the cache per chunk then that will lead to a lot more sys calls. ☝🏻

[ViRu-ThE-ViRuS]

what if you split the cache per event? as in all buf read events in one file, and all vim enter events in one file? Is this possible? Would it be better?

[lewis6991]

Lazy loading plugins via BufRead or VimEnter does NOT improve your startup time. You are just tricking the profiler. This is why your reported startup time is 3ms, which I was quite suspicious of. Even on an M1 Macbook my real startup time is ~100ms.

Lazy loading should only be done with events like: InsertEnter, CmdLineEnter, CursorHold and via keymaps, etc. Basically anything that isn't called during startup.

[ViRu-ThE-ViRuS]

[startup time plugin shows 7.7ms, which is the highest I have seen in my config]

 $ -> time vim +qall .

________________________________________________________
Executed in   68.70 millis    fish           external
   usr time   31.37 millis    0.13 millis   31.24 millis
   sys time   25.56 millis    1.35 millis   24.22 millis

 $ -> time vim +qall lua/autocommands.lua 

________________________________________________________
Executed in  198.52 millis    fish           external
   usr time  139.30 millis    0.14 millis  139.16 millis
   sys time   68.53 millis    1.46 millis   67.07 millis

Events related to Buffer like BufRead and so on are not loaded on empty startup nvim. I am mostly just opening an empty nvim instance and fuzzy navigating to the file I want to start editing.

In my setup, most of my plugins are loaded on BufRead, so that this style of using neovim becomes more responsive. I am aware that StartupTime plugins numbers are not the reality, and lazy loading is just tricking them, but this use case is effected by having the cache contain all plugins in one file.

Regardless, I now do understand that splitting the cache would just lead to more delays on top because of issuing multiple sys calls, which would slow down the more common use cases. thanks for clarifying my questions!

Amazing work on this plugin :)