performance on ubuntu on windows wsl2 extremely slow

[jcaplan]

steps to reproduce:

start zellij on Ubuntu instance in WSL2 on windows 10. Even with one pane and one tab, if I do anything (including hit enter or type a single character in the shell), I can see the cursor flickering across the entire terminal window as the pane re-renders. If I type quickly then the delay to render is really bad.

actual behaviour

I bashed 40 characters on the keyboard and timed that it took ~15 seconds from time I pressed the first character to when they finished rendering.

expected behaviour

lag should be undetectable.

[imsnif]

Hey, sorry about the experience. In our early alpha state we have not yet given performance a whole lot of work. That does sound rather slow though. Have you tried running it with --release?

[jcaplan]

yep #199.

[jcaplan]

fwiw it doesn't seem to be an inherent windows/wsl issue. If I open up a zellij instance inside a tmux pane the performance is much better.

[imsnif]

fwiw it doesn't seem to be an inherent windows/wsl issue. If I open up a zellij instance inside a tmux pane the performance is much better.

I'm not sure I understand the reasoning here...

[jcaplan]

well if there was something severe caused directly by wsl virtualization then I'd imagine the problem would persist with tmux. Somehow performance is improved with tmux and tmux itself runs very snappily. This implies to me that while there might be some virtualization overhead aggravating the issue on wsl where it is normally masked, something else is wrong.

I tried reducing all sleep times I could find and that helped ( Duration::from_millis(x), x < 5). This improved performance from unusable to just noticeable enough to be distracting (like a slightly slow ssh connection to remote terminal). Since this is something that tmux could not have improved, something else still has to be going on.

looking at CPU utilization:

zellij with one tab open:

tmux:

so that's a lot of CPU utilization considering nothing is happening. the next thing after that would be to investigate why things work better on tmux, if it's somehow intercepting buffers and optimizing the amount of data that ultimately gets written out. I still have to catch up a bit on terminal theory but is it always necessary to rewrite the whole screen buffer or just selectively update parts of it?

[imsnif]

As you see, there is definitely a lot of work to be done on performance. If you'd like to take that up, I would be happy to guide you to the right places at least concerning my ideas of what needs to be done.

About rendering: that's definitely one issue. Right now we render the entire screen for every change in one pane. There are a few issues we need to fix in order to be able to do this more intelligently, the major one being that we need to handle wide characters better. The issue: when there is a wide character in a pane (eg. some emojis), it sometimes "pushes" the right boundary of that pane, and so also the contents of all the panes to the left on that row. Re-rendering all panes hides this issue, but we should be able to handle these characters better. Other issues are more minor, but still need to be looked at.

About the issue at hand: my recommendation would be to first start by finding where exactly the lag is happening. You mentioned for example that this happens when you enter characters. What happens in Zellij when a character is entered is (roughly):

1. read from stdin
2. write that character to the pty of the active terminal
3. read the new state from the active terminal (this happens all the time regardless of input ofc)
4. render that state to screen

It could be that the lag you're seeing is happening in 1+2, for example. One way to know more is to maybe create some sort of bash loop inside a pane that writes the current timestamp to the terminal every 100ms and trying to see if there is any lag there. This takes 1+2 out of the equation. If you see near-tmux levels of performance there, then you will know better where to concentrate the performance improvement efforts. Makes sense?

[jcaplan]

I'll keep poking around :) I just checked on native Ubuntu, CPU utilization across all threads hovers around 7% which is quite high, but not enough on a native system to create a noticeable lag. With that said, it still feels very slightly laggy, and it is still improved when running inside a tmux pane.

it might actually be easier to debug on windows because for whatever the reason the situation is much more exaggerated but I don't think it's limited to windows.

[imsnif]

About the tmux pane: this is indeed interesting, but might be a red herring. eg. it could be happening because the pane is smaller. Does it still happen if you make the pane in tmux fullscreen?

[jcaplan]

good call! it totally depends on the size of the screen, which in hindsight is the most obvious contributor to the amount of data

[jcaplan]

Just pulled down changes since last week. #222 seems to have helped a lot!

There's still a big spike in CPU utilization when anything is changing (e.g. typing to a shell in a pane). I think the culprit is this:

            // while checking should_render rather than rendering each pane every time
            // is more performant, it causes some problems when the pane to the left should be
            // rendered and has wide characters (eg. Chinese characters or emoji)
            // as a (hopefully) temporary hack, we render all panes until we find a better solution
        // if self.should_render {
        if true {

in terminal_pane.rs. Is there a test case I can run? is it as simple as running a program to print a chinese character to stdout?