Module syntax-highlighting with prezto causes zsh to exit immediately

[mjbshaw]

I'm on OS X 10.10.5 using zsh 5.0.5 (x86_64-apple-darwin14.0). This happens in both iTerm2 and Apple's Terminal app. If I add the syntax-highlighting module to my .zpreztorc, so the module loading looks like this:

zstyle ':prezto:load' pmodule \
  'environment' \
  'terminal' \
  'editor' \
  'history' \
  'directory' \
  'spectrum' \
  'utility' \
  'completion' \
  'syntax-highlighting' \
  'prompt'

Here's me logging in with a pristine prezto install (I just cloned and set it up, so it's prezto at revision 165879d4b30dbf90d436b02a73786a28519d9e93):

And here's me logging in after enabling syntax-highlighting (adding 'syntax-highlighting' \ is the only change I've made to my .zpreztorc):

Note that zsh now immediately exits upon login.

[mjbshaw]

Looks like it's an issue in zsh 5.0.5. Running /usr/bin/zsh (which is v5.0.5) results in a segfault:

$ /usr/bin/zsh
[1]    89512 segmentation fault  /usr/bin/zsh

However, if I run zsh 5.1 there is not segfault and the syntax-highlighting module works just fine. I don't know if this is something that can be worked around or if it's not worth the energy given v5.1 works.

[sorin-ionescu]

Which highlights have you enabled?

[mjbshaw]

I haven't explicitly enabled any highlights, which I assume means just the main highlight should be loaded by default.

This is with a fresh clone of prezto (with the only modification being the 'syntax-highlighting' \ line in .zpreztorc to load the module; I didn't make any other modifications to change which highlights were used or anything like that), so I am just using whatever prezto with syntax-highlighting defaults to.

[sorin-ionescu]

No script should segfault Zsh. Can you upgrade Zsh? You might want to open an issue on the zsh-syntax-highlighting page.

[mjbshaw]

I can (and have) upgraded my zsh, but zsh 5.0.5 is what ships on OS X.

I'm not sure if this is a prezto issue or not. It doesn't crash if I completely remove prezto (and all of its ~/.z* config files), and make a clean .zshrc:

% cat .zshrc
source ~/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

(where ~/zsh-syntax-highlighting is a clean clone of zsh-syntax-highlighting). With this setup, zsh (v5.0.5) and zsh-syntax-highlighting both work correctly.

But as soon as I use prezto (with zsh 5.0.5) with syntax-highlighting, zsh crashes.

Additionally, if I change my prezto prompt theme to minimal (instead of sorin), then it does not segfault. In fact, I tried the themes agnoster, cloud, damoekri, giddie, kylewest, and minimal and they all worked. So far it looks like only the sorin theme causes the crash when syntax-highlighting is used.

[mjbshaw]

I just did a bit more investigating. Looks like the crash happens if line 93 of prompt_sorin_setup is executed. If I put a return 0 right before that line, then there is no crash. If I change $! to a fixed integer value (like 1), then zsh segfaults. If I set it to the PID of an existing process (I started vim in a new shell, and then hardcoded _prompt_sorin_precmd_async_pid to the PID of the vim process), zsh segfaults (and the vim process still executed, so it was never killed).

I can't tell exactly when the segfault occurs. It's certainly after the prompt_sorin_setup script runs.

[danielshahaf]

You might want to try zsh -f -o sourcetrace or zsh -x -f to trace what's being executed.

[ziegs]

It's specifically the call to zle reset-prompt on line 63, everything up to there executes.

[Eriner]

all that does is redraw the prompt. Unless zsh had a bug that caused that to bug out, that isn't the problem.

[danielshahaf]

Does it help if you change zle reset-prompt to zle -I?

[ziegs]

I'm not OP, but I do have the same exact problem as OP.

zle -I prevents segfault, but causes every <CR> to insert a blank prompt (see attached)

[Eriner]

using zle -I won't work; it doesn't take new directory changes into account like the redraw-prompt widget does.

[Eriner]

From a cursory reading, it seems like this fix: Changes between versions 5.0.8 and 5.1: Many changes to child-process and signal handling to eliminate race conditions and avoid deadlocks on descriptor and memory management.

may be why it bugs on earlier versions of zsh and not later versions. I cannot reproduce this, and therefore cannot debug it. The fact that it is segfaulting indicates it is a zsh problem.

[ziegs]

This is crashing on an earlier version than 5.0.8 for both myself and OP (zsh 5.0.5); It's the version both OS X and Ubuntu 14.04 LTS ship with. Agree that it's a zsh problem, but a recent pull of prezto+zsh-syntax-highlighting is triggering it. I'll try to find time to bisect to see if I can find what change on the prezto/syntax-highlighting side is making this happen.

[Eriner]

No, as far as I can tell, this is an async problem. The sorin and pure prompts use async.

That is exactly my point; this problem in zsh core was fixed in version 5.1.

[Eriner]

If you switch to a different prompt and the problem continues, then it indicates it is a prezto/syntax-highlighting problem. That said, I find that to be HIGHLY unlikely. This is very likely a memory corruption problem that happens when zsh tries to handle child processes, hence the segfault.

[danielshahaf]

using zle -I won't work; it doesn't take new directory changes into account like the redraw-prompt widget does.

Then perhaps call them both: if zle; then zle -I; zle reset-prompt; fi

[danielshahaf]

Another thing: try using zle .reset-prompt instead of zle reset-prompt — that'll bypass the zsh-syntax-highlighting wrapper (see _zsh_highlight_bind_widgets).

[ziegs]

OK. I'll play with it. This has been working fine for a while before I merged in from prezto's HEAD and updated submodules, though. It was using async before.

[ziegs]

@danielshahaf zle .reset-prompt seems to be working

[Eriner]

Oh, yes, that is understandable @zlegs. It is possible that there was something that changed (within the past 5 months that have passed since @sorin-ionescu updated it) that is now triggering this problem. That I understand. I'm just saying that the problem lies in zsh's process handling code, which was apparently fixed in 5.1. That explains why this was happening.

[danielshahaf]

@danielshahaf zle .reset-prompt seems to be working

In this case, try disabling zsh-syntax-highlighting and adding a dummy wrapper instead:

# 1. disable syntax highlighting

# 2. add a dummy wrapper:
f() { zle .$WIDGET "$@" }; zle -N reset-prompt f

For future reference, there's an experimental zsh-syntax-highlighting branch that will remove the need for the widget-wrapping dance: https://github.com/danielshahaf/zsh-syntax-highlighting/tree/mikachu-redrawhook/v2; it depends a patch to zsh itself: https://github.com/zsh-users/zsh/tree/mikachu/redrawhook (by @mikachu)

[ziegs]

@danielshahaf what's the difference between zle .reset-prompt and the function you posted? .reset-prompt is working as before, with syntax highlighting enabled

[danielshahaf]

@ziegs zle .reset-prompt bypasses the zsh-syntax-highlighting wrapper widget entirely. The function I posted is a dummy wrapper widget that forwards to the builtin widget. I wanted to see whether the bug was triggered by merely having a user-defined wrapper around the reset-prompt widget, or by something zsh-syntax-highlighting's reset-prompt wrapper does.

In theory, when reset-prompt hasn't been wrapped (as is the case when syntax highlighting has been disabled), installing the function I posted will be a no-op. However, I've seen cases where that invariant does not hold: https://github.com/zsh-users/zsh-syntax-highlighting/issues/183

[yriveiro]

This is happening in zsh 5.2 too

Any workaround for this?

[sorin-ionescu]

Is this a Prezto issue, a conflict with the editor module, or a zsh-syntax-highlighting issue? Prezto does not modify zsh-syntax-highlighting. It's an external module.

[c055]

https://github.com/sorin-ionescu/prezto/issues/1026#issuecomment-167540593

same issue here .. any workaround?

[danielshahaf]

To expand on @sorin-ionescu's question: Does this happen when starting zsh as zsh -f and in the new shell running source /path/to/zsh-syntax-highlighting.zsh? That will determine whether the cause is in z-sy-h or in the interaction of z-sy-h with other settings.

[yriveiro]

@danielshahaf, I followed the process that you described and it's working ...

[danielshahaf]

@yriveiro So that tells us it's not an issue in z-sy-h upstream; it might be an issue in prezto or in the combination of prezto and z-sy-h. I hope Sorin or someone else will be able to debug this further; I'm not a prezto user myself, I just maintain z-sy-h upstream.

[sorin-ionescu]

@danielshahaf I have difficulty debugging it since I do not experience it.

[danielshahaf]

@sorin-ionescu Understood, but since the problem doesn't occur without prezto loaded, neither I nor zsh upstream is likely to be able to help you reproduce or solve the problem...

[ghost]

I'm having the same problem on Ubuntu 12.04.5 LTS with zsh 4.3.17 whenever I enable 'syntax-highlighting' . zsh exits immediately. This is pretty frustrating and I have no idea how to solve this.

I'm using the following modules:

zstyle ':prezto:load' pmodule \
  'environment' \
  'terminal' \
  'editor' \
  'history' \
  'directory' \
  'spectrum' \
  'utility' \
  'ssh' \
  'completion' \
  'ruby' \
  'rails' \
  'git' \
  'syntax-highlighting' \
  'history-substring-search' \
  'prompt'

I'm also running it on OSX 10.11.2 with zsh 5.2 where I don't have any problems.

[thekondr]

Hi guys,

I was able to debug this issue.

The line that causes the issue is here.

We are trying to access $WIDGET which seems like for some reason is not available from asynchronously called function and results in a "crash".

I get the same issue with just echo $WIDGET line in TRAPUSR1 function in example from this page without prezto and syntax-highlighting enabled.

[dchenbecker]

It also seems to run fine on OSX with zsh 5.0.8 (homebrew), but I'm definitely seeing the issue on my Ubuntu 14.04 boxes. I'll see if upgrading to zsh 5.2 helps, but it would be nice if @thekondr has found a fixable problem.

[dchenbecker]

Confirmed that using zsh 5.2 on Ubuntu 14.04 works fine.

[kadoyau]

OSX 10.10.2 with zsh 5.2 has same issue.

[ghost]

OSX 10.10.5 with zsh 5.2 has same issue.

[jameskyle]

Same issue on Centos 7, Zsh 5.0.2

[belak]

I haven't been able to reproduce this on macOS 10.12.4 with zsh 5.2. If anyone manages to figure out exactly what's causing this issue, I'd be happy to look into it. It's just fairly hard to debug if I can't reproduce it.

[menghanl]

I got the same problem. And setting TERM=xterm-256color fixed it for me.

[belak]

What was TERM set to when it crashed? And what platform was it on?

[menghanl]

TERM was set to xterm before. On a customized version of ubuntu 14.04, with zsh 5.3.1.

[belak]

There have been some changes with how async tasks and zle reset-prompt are used in the sorin prompt. Can anyone here confirm this is still happening since I haven't been able to reproduce it yet? Thanks!

[c2h5oh]

Still happening with zsh 5.4.1 and prezto-git on Arch

[7h3kk1d]

I don't know if it's related but my gnome-terminal session completely crashes using the sorin theme sporadically on ubuntu 14.04 and zsh 5.0.2