Syntax highlighting is slow and CPU intensive

[DXist]

Scrolling of big ansible YAML configs (>100Kb) is slow and CPU intensive, slower then vim stock yaml highlighter and even slower than https://github.com/chase/vim-ansible-yaml.

[pearofducks]

Hmmm... if you have a config you're able to put up, I can take a look. However, highlighting here directly imports stock YAML, so I really doubt the few extra highlight rules are a showstopper.

The only thing I can think of is if your vim lacks +regexpengine.

Also, how in the world do you have > 100Kb files? That sounds like a maintenance nightmare. :)

[DXist]

Well, it's pretty maintainable yet. The file has a big dictionary with user information like:

erlich:
    # [[[ authorized
    authorized:
      ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyLcVa+XmVoquW9HxP64KKnKgNzUPai6/LE9Qb35OvgG3lfPVXmcJtL9+zu0WB+pttm+BzpKw8GZfHB8qvHNdChMGO9zU4FE32s8cthfUPMU309jI3Bt9Ll5ZXdgKBVN4s+6fhJTHw9qkLj1vuG6Qu2TgTludiG5YLA4nDPm6W1QgpR0WJizkyPFjz2IEiJWrRQw1+2UzvVo8KwsQavcbrzVB4s907tepSnT48TNJHGK3YvGv12pVZeJBZNRwZTgtFi4/e8TVFPMYf2x5edT+Uj1DRxYHdzww1vK7o0HMPlmKmmkAYj2Kgc1jSQEmX9OHNu/1ZnK++5Oj1XiYRJYeJ erlich@aviato
    # ]]]
    email: erlich.bachman@aviato.com
    home: /home/erlich
    name: Erlich Bachman
    shell: /usr/local/bin/bash

I wasn't able to reproduce CPU usage just by cloning this snippet. There was similar discussion at https://github.com/chase/vim-ansible-yaml/issues/11

In comparison to stock YAML highlighter ansible-vim used up to 100% CPU after moderate scrolling. yaml.vim used up to ~50%.

[pearofducks]

What's your output of vim --version?

Also what do you get when you run echo &regexpengine in Normal mode when you have a YAML file open?

Does performance improve if you set set synmaxcol=120 in your .vimrc?

[DXist]

vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Dec  9 2014 17:36:39)
Included patches: 1-488
Modified by pkg-vim-maintainers@lists.alioth.debian.org
Compiled by buildd@
Huge version without GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
-balloon_eval    +float           +mouse_urxvt     -tag_any_white
-browse          +folding         +mouse_xterm     +tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
-clientserver    -hangul_input    +netbeans_intg   +title
-clipboard       +iconv           +path_extra      -toolbar
+cmdline_compl   +insert_expand   +perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con      +lua             +rightleft       +windows
+diff            +menu            +ruby            +writebackup
+digraphs        +mksession       +scrollbind      -X11
-dnd             +modify_fname    +signs           -xfontset
-ebcdic          +mouse           +smartindent     -xim
+emacs_tags      -mouseshape      -sniff           -xsmp
+eval            +mouse_dec       +startuptime     -xterm_clipboard
+ex_extra        +mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    -xpm
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H     -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1     -I/usr/include/tcl8.6  -D_REENTRANT=1  -D_THREAD_SAFE=1  -D_LARGEFILE64_SOURCE=1
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -L/build/buildd/ruby2.1-2.1.2/debian/lib -fstack-protector -rdynamic -Wl,-export-dynamic -Wl,-E  -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim        -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl  -L/usr/lib -llua5.2 -Wl,-E  -fstack-protector -L/usr/local/lib  -L/usr/lib/x86_64-linux-gnu/perl/5.20/CORE -lperl -ldl -lm -lpthread -lcrypt -L/usr/lib/python2.7/config-x86_64-linux-gnu -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions  -L/usr/lib/x86_64-linux-gnu -ltcl8.6 -ldl -lz -lpthread -lieee -lm -lruby-2.1 -lpthread -lgmp -ldl -lcrypt -lm

echo &regexpengine returns 1. Setting it to 2 reduces pressure on CPU but visually sluggishness stays the same.

Decreasing synmaxcol does improve performance. Setting filetype to yaml improves performance.

[pearofducks]

I did some testing here, and I'm able to scroll a 2000-line .yaml file on a Macbook 12" with no slowdown, problems, or excessive CPU-usage.

My primary ideas are the lower synmaxcol, and to refactor your monolithic .yaml files to smaller files (this is in Ansible best-practice as well). I completely understand this could work against how you want to do things, but don't have any other options to offer right now.

Feel free to mess with the contents of syntax/ansible.vim and submit a pull request if you find a good solution!

[DXist]

Thank you for enlightening me about synmaxcol! Currently I use chase/vim-ansible-yaml plugin and find it pretty performant on this file so I can get shit done with it. I just tried your plugin as an alternative, found it less performant and decided to share this issue with you.

[CaptainQwark]

I had the same problem, but also for really small files with just a few lines e.g.:

---
- name: make users
  user: name={{ item.name }} comment="admin user created by ansible" password={{ item.password }} shell=/bin/bash groups=admin
  with_items:
    - { name: 'alex', password: '$6$DMm3AP8RA$Pi.PknhoDdEIQpYHvh1QUNPK.oHI6TbKBDGcHDLKNJz/NmKpTJq7gs.YgG0QJKlIxd92ucT9dURconhBuj5xA1' }
  tags: users

Vim was slow on just opening the file and walking the cursor through the lines (especially the curly brackets in the with_items list)

echo &regexpengine returned 0 on my vim. (stock vim on debian jessie).

adding this to my vimrc fixed the problem for me:

if exists('&regexpengine')
  set regexpengine=1
endif

I thought I'd just leave this here for future reference.

Thanks for your (now nicely working) plugin.

[pearofducks]

Thanks for noting your experience/fixes so hopefully it helps future users.

I just ran a test on 100% stock vim+jessie, and could not reproduce. Not only did I get echo &regexpengine as 1, but had no issues with slowdown in any way (and this was on a 5 USD DigitalOcean droplet).

Here's the commands I ran from a fresh install, and my vimrc, just for comparison!

_.bashhistory

apt-get install vim curl git
curl -fLo ~/.vim/autoload/plug.vim --create-dirs https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim
vim .vimrc (then entered the vimrc contents below)
vim playbook.yml

.vimrc

set nocp
call plug#begin('~/.vim/plugged')
Plug 'pearofducks/ansible-vim'
call plug#end()
filetype plugin indent on
syntax on
set encoding=utf-8