Inserting a new line in a large script might be slow

[icersong]

vim 8 insert new line, python filesize 1K

FUNCTION  <SNR>138_find_opening_paren()
    Defined: ~/.cache/vimfiles/plugins/vim-python-pep8-indent/indent/python.vim line 96
Called 4 times
Total time:   2.822975
 Self time:   0.728050

count  total (s)   self (s)
                                " optional arguments: line and column (defaults to 1) to search around
    4              0.000005     if a:0 > 0
    2              0.000009         let view = winsaveview()
    2              0.000009         call cursor(a:1, a:0 > 1 ? a:2 : 1)
    2              0.000007         let ret = s:find_opening_paren()
    2              0.000071         call winrestview(view)
    2              0.000002         return ret
    2              0.000001     endif

                                " Return if cursor is in a comment.
    2              0.000036     exe 'if' s:skip_search '| return [0, 0] | endif'

    2              0.000004     let nearest = [0, 0]
    8              0.000028     for [p, maxoff] in items(s:paren_pairs)
    6              0.000046         let stopline = max([0, line('.') - maxoff, nearest[0]])
    6   1.411188   0.727807         let next = searchpairpos( '\V'.p[0], '', '\V'.p[1], 'bnW', s:skip_special_chars, stopline)
    6              0.000036         if next[0] && (next[0] > nearest[0] || (next[0] == nearest[0] && next[1] > nearest[1]))
                                        let nearest = next
    6              0.000007         endif
    8              0.000008     endfor
    2              0.000003     return nearest

[toejough]

mine is even slower, and I've got a decent macbook:

FUNCTION  <SNR>102_find_opening_paren()
    Defined: ~/.vim/plugged/vim-python-pep8-indent/indent/python.vim line 96
Called 44 times
Total time:  22.053020
 Self time:   5.826575

count  total (s)   self (s)
                                " optional arguments: line and column (defaults to 1) to search around
   44              0.000057     if a:0 > 0
   22              0.000093         let view = winsaveview()
   22              0.000077         call cursor(a:1, a:0 > 1 ? a:2 : 1)
   22              0.000091         let ret = s:find_opening_paren()
   22              0.001036         call winrestview(view)
   22              0.000024         return ret
   22              0.000010     endif

                                " Return if cursor is in a comment.
   22              0.016860     exe 'if' s:skip_search '| return [0, 0] | endif'

   22              0.000053     let nearest = [0, 0]
   88              0.000312     for [p, maxoff] in items(s:paren_pairs)
   66              0.000462         let stopline = max([0, line('.') - maxoff, nearest[0]])
   66  11.006864   5.807683         let next = searchpairpos( '\V'.p[0], '', '\V'.p[1], 'bnW', s:skip_special_chars, stopline)
   66              0.000322         if next[0] && (next[0] > nearest[0] || (next[0] == nearest[0] && next[1] > nearest[1]))
                                        let nearest = next
   66              0.000078         endif
   88              0.000092     endfor
   22              0.000030     return nearest

[blueyed]

Thanks for the report/info in general.

I've looked into optimizing this a while back, but it is not trivial.

You can try changing https://github.com/Vimjas/vim-python-pep8-indent/blob/3e61be529bd193b640f015f440d659d05459580c/indent/python.vim#L51, which defines how far it looks for matching/opening parenthesis.

There are many tests at least, so you would be covered when trying to optimize it yourself. Unfortunately the logic / code is also not easy to follow (e.g. with :debug echo GetPythonPEPIndent(line(".")+1), but you might want to try it anyway).

[blueyed]

Also an example script would be useful in general (anonymized if needed, or just something that triggers this) - IIRC it might be especially slow with unmatched opening {.

I guess that it is (especially) slow for your with unclosed opening parenthesis as well?

[blueyed]

One idea might be to use a timeout with searchpairpos, but that should be done with https://github.com/vim/vim/commit/9d32276b52a63fccfae681f0d1d6ccb66efec1c0 (8.0.1483) only.

Related issue for Vim's own python indent script: https://github.com/vim/vim/issues/1098#issuecomment-429530484

[blueyed]

@toejough It appears to be similar slow for you btw, just that you have used it more (called 44 times instead of 4).

[blueyed]

To mitigate this you might want to try https://github.com/Vimjas/vim-python-pep8-indent/pull/118 (or just add the timeout yourself to the call).

I've found some older stashes in this regard, where I've experimented e.g. with doing a quick round first (only looking up e.g. 5 lines), before looking potentially 1000 lines up for {}.

Also caching might help in general.

Anyway, it appears to be related to your specific use case (the lines before o) - I usually have no problem with this.

See https://github.com/Vimjas/vim-python-pep8-indent/issues/99#issuecomment-439276756 for discussion about using the Python AST actually.

[blueyed]

Came across some bummer in Vim itself: it appears to evaluate the skip expression always.. :/ (note that this could maybe get optimized anyway, and likely should be handled manually for lower Vims anyway given this issue?!) (https://github.com/vim/vim/pull/3613)

[toejough]

wow, thank you for the prompt and informative responses!! 🥇

The updates in 118 (at least as of the last passing check I saw - efa7e6b0ee1448f98f5a359fe2b6a9b330434db7) cut the delay from this plugin in half. That's some significant improvement - thank you!

Happy to try out other updates if you'd like but what's in 118 is already significantly better (at least for me!)

[blueyed]

Great to hear, pushed some more. You can also try the red ones - tests are passing but the patch is not fully covered (i.e. the conceiled part).

[blueyed]

Merged https://github.com/Vimjas/vim-python-pep8-indent/pull/118.

Please open new issues with remaining things (and examples) for improvements.