Startup Performance hit with source zsh-syntax-highlighting

[xbeta]

I had this in my ~/.zshrc, and I found there are some performance issue when loading up this zsh.

source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

before comment out the above line:

 /usr/bin/time zsh -i -c exit;
        3.08 real         1.75 user         1.03 sys

after comment out the above line:

 /usr/bin/time zsh -i -c exit;
        2.69 real         1.47 user         0.89 sys

Current version at :

commit ad522a091429ba180c930f84b2a023b40de4dbcc

Note: I have other perf issues on startup that is not related to zsh-syntax-highlighting, but I'm pursuing each hotspot to cut the perf number lower. And zsh-syntax-highlighting is one of them.

[danielshahaf]

I had this in my ~/.zshrc, and I found there are some performance issue when loading up this zsh.

source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

To clarify, are you asking only about the startup performance, or also about performance during subsequent interactive usage?

What's the output of typeset -p ZSH_HIGHLIGHT_HIGHLIGHTERS ZSH_HIGHLIGHT_REVISION ZSH_HIGHLIGHT_VERSION ZSH_VERSION ZSH_PATCHLEVEL?

[xbeta]

@danielshahaf I should clarify a bit more, will update this issue ticket on the title. I'm talking specifically about startup performance.

Output from that command you suggested:

typeset -p ZSH_HIGHLIGHT_HIGHLIGHTERS ZSH_HIGHLIGHT_REVISION ZSH_HIGHLIGHT_VERSION ZSH_VERSION ZSH_PATCHLEVEL
typeset -a ZSH_HIGHLIGHT_HIGHLIGHTERS
ZSH_HIGHLIGHT_HIGHLIGHTERS=( main )
typeset ZSH_HIGHLIGHT_REVISION=HEAD
typeset ZSH_HIGHLIGHT_VERSION=0.6.0-dev
typeset ZSH_VERSION=5.2
typeset ZSH_PATCHLEVEL=zsh-5.2-0-gc86c20a

[danielshahaf]

It takes about the same time here:

% typeset -F SECONDS=0; source ./zsh-syntax-highlighting.zsh; echo $SECONDS 0.3567790000

I agree, it'd be good to make the startup faster. I don't intend to implement this myself, but patches would be welcome.

[xbeta]

Probably not related to this then. Closing it.

[z0rc]

Side note. There is zsh/zprof module that gives much more insight about startup time and most taxing function calls.

[danielshahaf]

@xbeta We both see the same overall time in z-sy-h, and it would be a good thing to reduce that time, so I'm reopening.

@z0rc I'm aware, thanks — and welcome aboard :).

[JamesYeoman]

I've followed https://xebia.com/blog/profiling-zsh-shell-scripts/ and here's the callee map from kcachegrind

As you can see, _zsh_highlight_bind_widgets (a function from zsh-syntax-highlighting) is the highest percentage single function in the callee map. While yes, there are plenty of things I could try to trim out, the fact that _zsh_highlight_bind_widgets takes 32,389 micros per call; which pushes zsh-syntax-highlighting-plugin.zsh up to 50,178 micros per call, which then pushes antigen's init.zsh up to 110,105 micros per call; is something that feels like it should be investigated.

For reference, here's Antigen's init.zsh list of callees

[danielshahaf]

Here's the contents of _zsh_highlight_bind_widgets as will be used as of the next zsh release:

https://github.com/zsh-users/zsh-syntax-highlighting/blob/ebef4e55691f62e630318d56468e5798367aa81c/zsh-syntax-highlighting.zsh#L437

As you can see, it literally does nothing. Furthermore, it's called only once. I rest my case :)

That codepath is only enabled by default as of zsh 5.8.0.3 and newer (not yet released) for reasons explained in the comment above the if. However, you can disable the is-at-least check locally if the interop considerations (point (2) in the link) don't affect you, in which case any zsh ≥5.3 will use the new codepath.

[JamesYeoman]

@danielshahaf this is the contents of _zsh_highlight_bind_widgets for me, and it's up to date with master (according to antigen)

_zsh_highlight_bind_widgets()
  {
    setopt localoptions noksharrays
    typeset -F SECONDS
    local prefix=orig-s$SECONDS-r$RANDOM # unique each time, in case we're sourced more than once

    # Load ZSH module zsh/zleparameter, needed to override user defined widgets.
    zmodload zsh/zleparameter 2>/dev/null || {
      print -r -- >&2 'zsh-syntax-highlighting: failed loading zsh/zleparameter.'
      return 1
    }

    # Override ZLE widgets to make them invoke _zsh_highlight.
    local -U widgets_to_bind
    widgets_to_bind=(${${(k)widgets}:#(.*|run-help|which-command|beep|set-local-history|yank|yank-pop)})

    # Always wrap special zle-line-finish widget. This is needed to decide if the
    # current line ends and special highlighting logic needs to be applied.
    # E.g. remove cursor imprint, don't highlight partial paths, ...
    widgets_to_bind+=(zle-line-finish)

    # Always wrap special zle-isearch-update widget to be notified of updates in isearch.
    # This is needed because we need to disable highlighting in that case.
    widgets_to_bind+=(zle-isearch-update)

    local cur_widget
    for cur_widget in $widgets_to_bind; do
      case ${widgets[$cur_widget]:-""} in

        # Already rebound event: do nothing.
        user:_zsh_highlight_widget_*);;

        # The "eval"'s are required to make $cur_widget a closure: the value of the parameter at function
        # definition time is used.
        #
        # We can't use ${0/_zsh_highlight_widget_} because these widgets are always invoked with
        # NO_function_argzero, regardless of the option's setting here.

        # User defined widget: override and rebind old one with prefix "orig-".
        user:*) zle -N $prefix-$cur_widget ${widgets[$cur_widget]#*:}
                eval "_zsh_highlight_widget_${(q)prefix}-${(q)cur_widget}() { _zsh_highlight_call_widget ${(q)prefix}-${(q)cur_widget} -- \"\$@\" }"
                zle -N $cur_widget _zsh_highlight_widget_$prefix-$cur_widget;;

        # Completion widget: override and rebind old one with prefix "orig-".
        completion:*) zle -C $prefix-$cur_widget ${${(s.:.)widgets[$cur_widget]}[2,3]}
                      eval "_zsh_highlight_widget_${(q)prefix}-${(q)cur_widget}() { _zsh_highlight_call_widget ${(q)prefix}-${(q)cur_widget} -- \"\$@\" }"
                      zle -N $cur_widget _zsh_highlight_widget_$prefix-$cur_widget;;

        # Builtin widget: override and make it call the builtin ".widget".
        builtin) eval "_zsh_highlight_widget_${(q)prefix}-${(q)cur_widget}() { _zsh_highlight_call_widget .${(q)cur_widget} -- \"\$@\" }"
                 zle -N $cur_widget _zsh_highlight_widget_$prefix-$cur_widget;;

        # Incomplete or nonexistent widget: Bind to z-sy-h directly.
        *)
           if [[ $cur_widget == zle-* ]] && (( ! ${+widgets[$cur_widget]} )); then
             _zsh_highlight_widget_${cur_widget}() { :; _zsh_highlight }
             zle -N $cur_widget _zsh_highlight_widget_$cur_widget
           else
        # Default: unhandled case.
             print -r -- >&2 "zsh-syntax-highlighting: unhandled ZLE widget ${(qq)cur_widget}"
             print -r -- >&2 "zsh-syntax-highlighting: (This is sometimes caused by doing \`bindkey <keys> ${(q-)cur_widget}\` without creating the ${(qq)cur_widget} widget with \`zle -N\` or \`zle -C\`.)"
           fi
      esac
    done
  }

[danielshahaf]

@JamesYeoman What's your question? Did you read my previous comment?

[JamesYeoman]

Ah, sorry, I glossed over that second paragraph. Disregard what I said then