search

zellij-org / zellij

A terminal workspace with batteries included
https://zellij.dev
MIT License
21.19k stars 643 forks source link

Server: Buffer full #2041

Closed ovove closed 1 year ago

ovove commented 1 year ago

Staring with Zellij 0.34.4 I get occasional crashes. I am unfortunately not able to come up with a reproducible way recreate it. Whenever I get the crash, the only thing I see is this message

Error occurred in server: Buffer full

Basic information

zellij --version: 0.34.4 stty size: 46 178 uname -av: Darwin oMac 22.2.0 Darwin Kernel Version 22.2.0: Fri Nov 11 02:08:47 PST 2022; root:xnu-8792.61.2~4/RELEASE_X86_64 x86_64

List of programs you interact with as, PROGRAM --version: output cropped meaningful, for example: zsh --version: zsh 5.8.1 (x86_64-apple-darwin22.0) emacs --version: GNU Emacs 28.2 clang --version: clang version 15.0.6

Further information n/a

raphCode commented 1 year ago

This usually happens when the client stalls and can't consume all the messages the server sent in a timely matter. Can this be related to high system load in your case? So to speak this message is a "feature" that was introduce to stop zellij from hanging once a client doesn't respond.

Luckily you can reattach to your session afterwards.

ovove commented 1 year ago

Not sure about the load at the time of the crash. I have an 8 core macbook air, that I mostly use for some simple development during my free time. Nothing serious though. I'd be surprised if the load average goes much higher than 8-ish. But for sure, I can try to keep a close I to the load average and see if I can correlate it to any zellij crashes,

imsnif commented 1 year ago

Also, which terminal emulator are you using?

ovove commented 1 year ago

I'm using iTerm2

danillos commented 1 year ago

It is happening to me too. Iterm2 + Helix.

The editor was slow, but I'm pretty sure is not a resource issue.

I moved to Kitty and no issues so far.

Just an update: It happened with Alacritty too.

aniketd commented 1 year ago

This potentially causes loss of done work. I have seen this happen twice within the last 2 days.

uname -av: Linux nixos 5.15.85 #1-NixOS SMP Wed Dec 21 16:36:38 UTC 2022 x86_64 GNU/Linux
 terminal: Gnome Console
    panes:
      - 1 helix-editor
      - 4 other zsh sessions running programs that can generate a lot of output very quickly.
imsnif commented 1 year ago

Until I issue a fix for this (need to investigate a little further) I can recommend as a workaround to use a GPU accelerated terminal. This likely happens because the terminal emulator doesn't read STDOUT fast enough, Using something like Alacritty should help a lot.

fbernier commented 1 year ago

I've also been experiencing this on my ryzen desktop since the last release.

using:

sleepytraveler commented 1 year ago

I run into this a lot where I'm running zellij on a remote machine I'm connected over ssh. Local terminal emulator can be kitty or wezterm - both hit the issue. Local shell and remote shell are fish. When a build command starts spewing a number of lines I get the "Buffer full" error.

Machine is running: Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux Haswell CPU with 16 cores running at 2.1GHz.

imsnif commented 1 year ago

Just FYI, there's a fix for this in the works: https://github.com/zellij-org/zellij/pull/2068

alex35mil commented 1 year ago

Alacritty. Also seeing it relatively often. Mostly when something produces a lot of output.

matu3ba commented 1 year ago

I experienced the very same problem with a pile of queued > events over ssh connection to resize a neovim window. (using hydra.nvim)

raphCode commented 1 year ago

Using ssh might increase the change to hit the error. As an workaround, try mosh inplace of ssh.

sleepytraveler commented 1 year ago

@raphCode - thank you for the suggestion to use mosh.

I am happy to report that while using mosh, the number of times I hit this issue is greatly reduced. Hopefully any one else running into this issue while using zellij on a remote system over ssh finds this discussion.

alex35mil commented 1 year ago

FWIW I do everything on local machine and still hit this relatively often. And I think quite often it happens when I scroll very fast using my mouse.

raphCode commented 1 year ago

Just connect with mosh to localhost :clown_face:

The PR was merged, so it might be better in the next release...

bluss commented 1 year ago

Using zellij 0.34.4 I can reproduce the "Buffer full" error using the following command:

/bin/ls -l --hyperlink=always -R

It reaches around line 35000 before it crashes (which happens quickly) it emits the "Buffer full" message

If I drop the hyperlink option it lists all 450 000 lines correctly.

ls version: ls (GNU coreutils) 8.30 Terminal: GNOME Terminal 3.36.2 using VTE 0.60.3 +BIDI +GNUTLS +ICU +SYSTEMD

matu3ba commented 1 year ago

It reaches around line 35000 before it crashes (which happens quickly)

If I get buffer full, then the server becomes completely unresponsive, so I need to manually kill it with kill -SIGKILL $(pgrep zellij). Do you have the same problem?

imsnif commented 1 year ago

@matu3ba - can you help us consistently reproduce the case where the server becomes unresponsive? @har7an worked on fixing this issue (the fix is now in main) but could not reproduce this behavior. For us every time this happened we were able to reattach.

hudon commented 1 year ago

I was able to reproduce this error consistently, with this setup:

I run zellij nested like this bc I use it locally for tabs/panes, and remotely for sessions.

Once it's setup, I run ls -l /usr/bin (which has about 910 items) and scroll the mouse at the same time. This crashes the nested (ie. remote) zellij server every time with "buffer full".

I noticed that when I don't run it nested (ie. I SSH into my server without running zellij locally first), the crash does not happen.

matu3ba commented 1 year ago

can you help us consistently reproduce the case where the server becomes unresponsive?

No, I tried to reproduce it, but I was not able to do so yet.

I would emphasize to use Windows Terminal for WSL, which can handle incoming data much faster as that also was leading to buffer full for me. It is available in the Microsoft store. The other shells appear to have an incomplete osc52 implementation, which works on zellij, but does not work for nvim-osc52.

Mosh is allowed to silently drop data (not even creating an offline queue on server or client etc with fallback slow but reliable mode), which I think is unacceptable. Aside, it does not support osc52 https://github.com/mobile-shell/mosh/issues/637. Dropping data or slowness is a fundamental shell problem having no separate control and data channel. Mosh could have improved that due to controlling server and client, but unfortunately they did not.

hudon commented 1 year ago

Installed 0.35.2 on both local and remote, and I can no longer reproduce this bug https://github.com/zellij-org/zellij/issues/2041#issuecomment-1426225642. Thank you so much!!

ovove commented 1 year ago

I'm now at Zellij 0.36.0 and I haven't seen the reported issue for quite some time now. So perhaps this ticket can be closed now?

matu3ba commented 1 year ago

I'm now at Zellij 0.36.0 and I haven't seen the reported issue for quite some time now. So perhaps this ticket can be closed now?

Me too. Could be nice, if it is confirmed, that this specific buffer full error is logged properly to prevent silent/hidden regressions.

har7an commented 1 year ago

@matu3ba If this specific error does reoccur, it will now properly terminate the offending clients connection and print a pretty verbose error message (See https://github.com/zellij-org/zellij/pull/2068#issue-1524385000). The server will keep running in the background and can usually be reattached to.

I'll close the issue now, feel free to reopen it in case it does occur again. Happy to hear it's fixed!