Undoing large operations on large file locks vim up

[marcopaganini]

I'm evaluating vim-orgmode for personal use and discovered that undoing a large operation on a large file (~750KB, ~19k lines) locks vim up completely. Using top and strace on another screen, I can see vim consuming an entire core and doing a considerable number of mmap operations.

I started to trim the file to a smaller size, and the problem happens even with 5k lines files. Files with 1k lines will be slow, but not lock up. I waited for about 5 minutes for the undo operation to complete before giving up.

Info:

$ vim --version | head -5
VIM - Vi IMproved 8.1 (2018 May 18, compiled Feb 22 2019 12:55:04)
Included patches: 1-875, 878, 884, 948
Modified by team+vim@tracker.debian.org
Compiled by team+vim@tracker.debian.org
Huge version with GTK3 GUI (...)

[jceb]

Hm, sorry to hear that. I noticed that vim-orgmode is very slow for larger files. However, for the moment I wont investigate this issue any further because of time limitations.

[schlagenhauf]

I'm encountering the same problem and its so disruptive to my work that i even considered switching to Spacemacs. Before i do that, i'd try my luck with improving the performance of vim-orgmode. @jceb are there any pointers you can give where to start? Is there a main culprit one should tackle?

[jceb]

@schlagenhauf the fix that will immediately give you great speed is to just use the syntax highlighting and nothing else, not even the indentation script.

I didn't track the issue down and can't give you clear directions. I'm afraid you need to start measuring if you want to find and fix the issue.

[schlagenhauf]

Thanks for the hint, i'll try that.

I'm new to Vim plugin development but i did some superficial tests. Using a dummy org file with 7006 lines, profiling :e dummy.org produces (ignore nvim / dein paths):

FUNCTION  GetOrgFolding()
    Defined: ~/.cache/dein/.cache/init.vim/.dein/indent/org.vim line 41
Called 7006 times
Total time:   8.141392
--
FUNCTION  provider#python3#Call()
    Defined: /usr/share/nvim/runtime/autoload/provider/python3.vim line 26
Called 7300 times
Total time:   8.271677

(omitting all other calls that were way below 0.x total time)

I then used the scenario used in test_pluging_misc.py: test_fold_heading_start(self) and timed calls to fold_orgmode() (100k loops):

Computation time (first): 6.927 ms
Computation time (total): 22843.524 ms
Computation time (mean per loop): 0.228 ms

I assume the per-call speedup is due to caching. I'm not yet versed enough to tell how often this is called when using vim-orgmode normally, but assuming fold_orgmode() is called for every line after an undo, we quickly reach seconds of computation time. If the whole document needs to be parsed for every call, we get O(n^2) complexity in the number of lines which would explain the lockup that @marcopaganini encountered.

I'll keep looking.