Excessive CPU usage

[cewood]

I recently noticed that my Neovim installation was using a bit of CPU, actually a lot when multiplied by having many buffers/windows open. After some trial and error disabling large chunks of my config, I narrowed it down to vimade and the python3 client.

By excessive usage I am referring to 7-10% with a single buffer open doing nothing, and 40-50% with many buffers/windows open. For comparison, with the same set of many buffers/windows open and vimade disabled the usage has completely disappeared; the python3 client typically isn't running, and the python2 client has zero cpu usage on average.

I'm using the 211478c commit of vimade, with the following version of Neovim:

NVIM v0.3.5
Build type: Release
LuaJIT 2.0.5
Compilation: /usr/bin/cc -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -Wconversion -O2 -DNDEBUG -DMIN_LOG_LEVEL=3 -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wimplicit-fallthrough -Wvla -fstack-protector-strong -fdiagnostics-color=auto -Wno-array-bounds -DINCLUDE_GENERATED_DECLARATION
S -D_GNU_SOURCE -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -I/build/neovim/src/build/config -I/build/neovim/src/neovim-0.3.5/src -I/usr/include -I/build/neovim/src/build/src/nvim/auto -I/build/neovim/src/build/include
Compiled by builduser

Features: +acl +iconv +jemalloc +tui

and Python 3.7.3 with the following package versions:

msgpack    0.6.1  
neovim     0.3.1  
pynvim     0.3.2

Please let me know if there's any other info you need and I'll do my best to provide it.

[TaDaa]

The excessive usage you see is it the python process - I was able to reproduce performance problems there depending on my g:vimade.checkinterval value; if the value is too low the cpu usage will spike and stay there. I'll have to dig into this more tonight.

For now there are two ways to fix this -- a) Set checkinterval to a higher value :

let g:vimade={}
let g:vimade.checkinterval=1000

b) Use cursorhold instead of the async timer:

let g:vimade={}
let g:vimade.usecursorhold=1

The async checks are mainly for scrolling non-active buffers in gui vim/nvim. So if you are using terminal it shouldn't make a difference. I'll also change the nvim defaults when I have a chance.

[TaDaa]

For a long term solution I think I will move the state checks into vimscript so that Python is called less frequently, which should be a better solution

[cewood]

The excessive usage you see is it the python process

Yes, it was the python3 client specifically, and I am using Neovim without a gui (in a terminal).

I'll try out the two solutions you posted and let you know how I go. Thanks for looking into this and the prompt reply.

[cewood]

After trying both the options, I found that option B yielded the best results for me in combination with 1192d6f. With option A alone I still noticed a consistent 3-5% of CPU usage from the python client, which was enough to push my load average above 1.0, whereas it's usually comfortably hovering around ~0.66.

For a long term solution I think I will move the state checks into vimscript so that Python is called less frequently, which should be a better solution

This sounds like a good idea, I'll keep an eye out if this makes it in later.

Thanks again for your help, it's greatly appreciated.

[ttm436]

Chiming in here, I have seen a large increase in performance after this commit.

[TaDaa]

@ttm436 good to hear thanks. @cewood let me know if you still are having issues if not I'll close this issue this weekend.

There will be more performance boosts coming (relatively soon) but need to address a couple bugs first.