Slower performance in Neovim (than Vim) when fuzzy finding big trees like Linux kernel source.

[bluz71]

Firstly, I love this plugin, the speed and quality benefits are obvious, many thanks. I blogged about CtrlP and this plugin recently.

Anyway, doing some performance testing I noticed a difference in speed between Neovim (0.2.0) and Vim (8.0) when dealing with very large trees such as the Linux source. I don't think it is an issue with cpsm, it feels like it may be with Neovim itself. But before I report an issue with the Neovim crew it would be good to gather some info (if possible).

Setup, Vim 8.0 and Neovim 0.2.0 along with CtrlP and cpsm plugins. Note, I also use ripgrep for the ctrlp user command:

let g:ctrlp_user_command  = 'rg %s --files --color=never --glob ""'

Then checkout the Linux kernel, which will be in the order of 58,000 files. Launch Vim/Neovim, then CtrlP then hold j key for a while, then backspace, rinse and repeat. In Vim the key hold is super responsive whilst in Neovim it will really lag after a while. I see this on both a slower Macbook and fast Linux desktop.

With smaller trees, such as Ruby on Rails source with 3,500, no such obvious lagginess is seen. Big source trees like Linux are needed.

I tested with a minimal Neovim setup and I still see the lag. Any ideas why Neovim has worse performance with CtrlP + cpsm with huge trees? Does anyone else see this?

Also, I am definitely using cpsm on both Vim and Neovim since the ranked results are the same in both (which differ from CtrlP default).

Lastly since this is an informational request please don't hesitate to close this request. Thanks.

[bluz71]

It just so happened that a related issue was discussed on Reddit /r/neovim yesterday.

It boils down to this existing Neovim issue: legacy python provider is slow #7063. It may be worthwhile for you to read the whole thread, it very much relates to cpsm performance with Neovim.

Basically, in Vim the Python interface runs in-process whilst in Neovim the Python interface is via RPC. That difference is the cause of cpsm lagginess in Neovim.

To fix this Neovim + cpsm lag maybe one day cpsm should have a Neovim specific fork where the Python part of cpsm is replaced with Neovim's in-process Lua?

I am closing this issue since an existing Neovim issue exists #7063.

P.S. It would still be very nice for cpsm to be fast in Neovim just like it is in Vim.

[cjappl]

hi @bluz71, did you ever get to the bottom of this? Or find a work around? Your blog post led me to cpsm, but it is worse than the normal ctrlp search by a big margin on large codebases

[bluz71]

I switched to fzf. I haven’t used CtrlP in about four years.

[cjappl]

Haha! Well, maybe I’ll look into that then. Have a blog post describing your set up??

Thanks for the quick response.

On Mon, Jul 26, 2021 at 4:17 PM, Dennis B @.***> wrote:

I switched to fzf. I haven’t used CtrlP in about four years.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[Shougo]

The real reason is cpsm uses Python interface(if_python) and it is slow in neovim. It is the architecture problem in neovim. It cannot be fixed in cpsm side.

[bluz71]

Haha! Well, maybe I’ll look into that then. Have a blog post describing your set up??

Yes, Fuzzy Finding in Bash with fzf and Fuzzy Finding in Vim with fzf.

There are also now a couple native Lua-based fuzzy finding plugins for Neovim 0.5:

• telescope.nvim
• Snap

[bluz71]

The real reason is cpsm uses Python interface(if_python) and it is slow in neovim. It is the architecture problem in neovim.

I am not sure it is an architectural problem per-se. Python in Neovim is a remote procedure call (RPC). RPC by their very nature are slow.

This is a problem that will likely never be fixed. Neovim issue 7063 'Python provider is slow' is now locked as resolved.

My understanding is the Python in Vim will eventually also be an external invocation (not directly running inside Vim); Vim9script will be preferred (I am lead to believe).

To be honest, Python invocation within Neovim is somewhat considered a legacy interface anyway. The reason why folks used Python in Vim plugins is primarily because Vimscript is very very slow.

The solution to that in Neovim is to use the Lau API and inbuilt LuaJIT runtime of Neovim.

[Shougo]

The main problem is ctrlp is implemented in Vim script and cpsm is called by Python. It needs if_python and it is slow in neovim.

Vim uses directly library call instead of RPC. So Vim8 is faster than neovim.

[bluz71]

Vim uses directly library call instead of RPC. So Vim8 is faster than neovim.

Vim 9 may well end up similar to Neovim. The direct in-process Python bindings, it appears, will be phased out for Vim 9, see here.

[Shougo]

Yes. It will be legacy.

[cjappl]

Haha! Well, maybe I’ll look into that then. Have a blog post describing your set up??

Yes, Fuzzy Finding in Bash with fzf and Fuzzy Finding in Vim with fzf.

There are also now a couple native Lua-based fuzzy finding plugins for Neovim 0.5:

* [telescope.nvim](https://github.com/nvim-telescope/telescope.nvim)

* [Snap](https://github.com/camspiers/snap)

Thanks for this @bluz71. If you're a fan of fzf, you may like this git wrapper for it that I work on.

Cheers!