wez / wezterm

A GPU-accelerated cross-platform terminal emulator and multiplexer written by @wez and implemented in Rust
https://wezfurlong.org/wezterm/
Other
16.77k stars 751 forks source link

Crash upon RDP disconnect reconnection perhaps due to screen resize #265

Closed hgkamath closed 3 years ago

hgkamath commented 4 years ago

priority: important, affects usability

Describe the bug

Wezterm is started in a RDP session and is put into a left-tiled/right tiled or maximized state, the RDP session is disconnected. When the RDP session is subsequently reconnected, wezterm will have crashed. edit: The regular resizable window is also affected, but it will be in a frozen state, movable, alt-tab-able, until something triggers it to crash. This is perhaps caused by the 'screen resize adjustments'/'allocation of fresh screen' as the RDP session ends between remote and local machines Happens when Wezterm is started inside the RDP session, and not when wezterm is already started before the RDPsession. ergo, a workaround exists and is described later.

Use case: Using wezterm in an RDP session may be a common use case

Environment (please complete the following information):

The RDP-remotemachine Machine2 has a resolution of 2160x1440 The RDP local machine Machine1 has a resolution of 1920x1080 No desktop dpi-scaling (desktop font scaling) in either machine (this is set in Settings/Display) Note that in my case, the wezterm started inside RDP discovers a larger screen-size upon disconnect.

To Reproduce

Steps to reproduce the behavior (1)

  1. From Machine 1, RDP to remote machine (Machine2)
  2. Start wezterm
  3. Tile left/ Tile right/maximize
  4. Disconnect RDP
  5. Reconnect RDP
  6. Wezterm will have in all likelihood crashed

Steps to reproduce the behavior (2)

  1. From Machine 1, RDP to remote machine (Machine 2)
  2. Start wezterm, Let it remain in normal window mode, any size
  3. Disconnect RDP
  4. Reconnect RDP
  5. Now attempt to tile/maximize
  6. Wezterm will have in all likelihood crashed

Configuration

Nothing unusual

local wezterm = require 'wezterm';

local default_prog;
local set_environment_variables = {}

if wezterm.target_triple == "x86_64-pc-windows-msvc" then
  set_environment_variables["GROOT"] = "E:\\groot"
  -- set default_prog
  -- command prompt + inject clink into the command prompt
  -- default_prog = {"cmd.exe", "/s", "/k", "c:/clink_0.4.9/clink_x64.exe", "inject", "-q"}
  -- powershell
  default_prog = {"pwsh.exe"}
end

return {
  -- Default shell
  default_prog = default_prog ,
  set_environment_variables = set_environment_variables,

  color_scheme = "Material_GK2",
  -- Example light themes:
  --   "3024 Day", "ayu_light", "Belafonte Day", "CLRS", "AtomOneLight", "Github", "Man Page", "Material",
  --   "OneHalfLight", "Night Owlish Light", "Novel", "PencilLight", "Spring", "Tango Adapted",
  --   "Terminal Basic", "Violet Light"
  -- color_scheme = "Builtin Solarized Light",
  -- color_scheme = "Builtin Solarized Dark",

  -- The font size, measured in points
  -- default: font_size = 10.0,
  font_size = 11.0,

  -- The font family name.  The default is "Menlo" on macOS,
  -- "Consolas" on Windows and "monospace" on X11 based systems.
  -- You may wish to download and try either "JetBrains Mono" or
  -- "Fira Code" to enjoy ligatures without buying an expensive font!
  -- font = wezterm.font("Operator SSm Lig Medium"),
  -- font = wezterm.font("Source Code Pro Medium"),
  -- font = wezterm.font("Courier New"),
  -- font = wezterm.font("Fira Code"),
  font = wezterm.font("Consolas"),
  font_antialias = "Subpixel",
  font_hinting = "Full",
  font_shaper = "Harfbuzz",

  -- The DPI to assume, measured in dots-per-inch
  -- This is not automatically probed!  If you experience blurry text
  -- or notice slight differences when comparing with other terminal
  -- emulators, you may wish to tune this value!
  -- dpi = 96.0,
  dpi = 96.0,

  -- font_rules={
    -- {
    --   intensity="Bold",
    --   font=wezterm.font("Courier New", {
    --     -- you can override the default bold text color with this
    --     foreground="tomato",
    --   })
    -- },
    -- {
    --   italic=true,
    --   font=wezterm.font("Consolas Italic", {
    --     -- you can override the default bold text color with this
    --     foreground="tomato",
    --   })
    -- },
  -- },
  -- ratelimit_output_bytes_per_second=4289999998,
}

Expected behavior

Wezterm should handle RDP screen-size changes and more importantly never crash

Screenshots

NA

Session Recording

NA, can't do wt-record in windows

If the issue is with the way that escape sequences are processed it can be helpful to capture the terminal output using the wt-record script to run wezterm and record a transcript. This requires the script utility to be installed on your system (this is part of macOS and available in the util-linux package on linux systems).

In the example below a file named 20180225161026.tgz is produced. Please attach that file to this issue, or if it contains private or sensitive issue that you don't want the public to see on GitHub, please find some other way to get that file to a project contributor (perhaps Dropbox or email?).

$ ./wt-record
Transcript recorded in 20180225161026.tgz

You can use wt-replay 20180225161026.tgz to replay that file.

wt-record can only record the terminal output; it cannot record the input events going in to the terminal, so if you are having an issue with input, please be sure to describe it below!

RUST_LOG

with set RUST_LOG=info

...
}
 2020-09-12T18:31:23.186Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x1ac, handle_type: Unknown } }
 2020-09-12T18:31:23.204Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-12T18:31:23.273Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-12T18:31:23.300Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-12T18:31:23.313Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false

C:\Users\gana>thread 'main' panicked at 'failed to advise of resize: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer', src\frontend\gui\termwindow.rs:1839:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 2020-09-12T18:31:44.710Z ERROR window::os::windows::window         > caught Any

with set RUST_BACKTRACE=1

...
}
 2020-09-12T19:14:30.601Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x260, handle_type: Unknown } }
 2020-09-12T19:14:30.621Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-12T19:14:30.770Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-12T19:14:30.800Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-12T19:14:30.812Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false

C:\Users\gana>
C:\Users\gana>
C:\Users\gana>
C:\Users\gana>
C:\Users\gana>
C:\Users\gana>thread 'main' panicked at 'failed to advise of resize: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer', src\frontend\gui\termwindow.rs:1839:14
stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: CallWindowProcW
  15: CallWindowProcW
  16: glPushClientAttrib
  17: CallWindowProcW
  18: DispatchMessageW
  19: SendMessageTimeoutW
  20: KiUserCallbackDispatcher
  21: NtUserMessageCall
  22: SendMessageW
  23: GetWindowTextW
  24: IsIconic
  25: Ordinal132
  26: Ordinal132
  27: GetWindowTextW
  28: <unknown>
  29: CallWindowProcW
  30: CallWindowProcW
  31: glPushClientAttrib
  32: CallWindowProcW
  33: DispatchMessageW
  34: LookupIconIdFromDirectoryEx
  35: KiUserCallbackDispatcher
  36: NtUserPeekMessage
  37: PeekMessageW
  38: PeekMessageW
  39: <unknown>
  40: <unknown>
  41: <unknown>
  42: <unknown>
  43: <unknown>
  44: <unknown>
  45: <unknown>
  46: BaseThreadInitThunk
  47: RtlUserThreadStart
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
 2020-09-12T19:14:47.212Z ERROR window::os::windows::window         > caught Any

with set RUST_BACKTRACE=full (seems identical to previous)

...
}
 2020-09-12T19:16:18.095Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x258, handle_type: Unknown } }
 2020-09-12T19:16:18.111Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-12T19:16:18.163Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-12T19:16:18.195Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-12T19:16:18.206Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false
thread 'main' panicked at 'failed to advise of resize: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer', src\frontend\gui\termwindow.rs:1839:14
stack backtrace:
   0:     0x7ff6e47562de - <unknown>
   1:     0x7ff6e477787c - <unknown>
   2:     0x7ff6e474c863 - <unknown>
   3:     0x7ff6e475973b - <unknown>
   4:     0x7ff6e4759388 - <unknown>
   5:     0x7ff6e4759f40 - <unknown>
   6:     0x7ff6e4759a8f - <unknown>
   7:     0x7ff6e47753d0 - <unknown>
   8:     0x7ff6e4775213 - <unknown>
   9:     0x7ff6e404a484 - <unknown>
  10:     0x7ff6e404a6ea - <unknown>
  11:     0x7ff6e403eeda - <unknown>
  12:     0x7ff6e4430f89 - <unknown>
  13:     0x7ff6e443377d - <unknown>
  14:     0x7ffbfc8ae858 - CallWindowProcW
  15:     0x7ffbfc8ae4ee - CallWindowProcW
  16:     0x7ffbbeb6f150 - glPushClientAttrib
  17:     0x7ffbfc8ae858 - CallWindowProcW
  18:     0x7ffbfc8ae3dc - DispatchMessageW
  19:     0x7ffbfc8c0bc3 - SendMessageTimeoutW
  20:     0x7ffbfd14fbf4 - KiUserCallbackDispatcher
  21:     0x7ffbfb041124 - NtUserMessageCall
  22:     0x7ffbfc8adf33 - SendMessageW
  23:     0x7ffbfc8acbc3 - GetWindowTextW
  24:     0x7ffbfc8ac2af - IsIconic
  25:     0x7ffbf7dbd7e9 - Ordinal132
  26:     0x7ffbf7dbd1d1 - Ordinal132
  27:     0x7ffbfc8ac7e3 - GetWindowTextW
  28:     0x7ff6e443379a - <unknown>
  29:     0x7ffbfc8ae858 - CallWindowProcW
  30:     0x7ffbfc8ae4ee - CallWindowProcW
  31:     0x7ffbbeb6f150 - glPushClientAttrib
  32:     0x7ffbfc8ae858 - CallWindowProcW
  33:     0x7ffbfc8ae3dc - DispatchMessageW
  34:     0x7ffbfc8c5c00 - LookupIconIdFromDirectoryEx
  35:     0x7ffbfd14fbf4 - KiUserCallbackDispatcher
  36:     0x7ffbfb041064 - NtUserPeekMessage
  37:     0x7ffbfc8aa5c3 - PeekMessageW
  38:     0x7ffbfc8aa523 - PeekMessageW
  39:     0x7ff6e4427968 - <unknown>
  40:     0x7ff6e41cac16 - <unknown>
  41:     0x7ff6e41c6fa0 - <unknown>
  42:     0x7ff6e3f9fc16 - <unknown>
  43:     0x7ff6e475a206 - <unknown>
  44:     0x7ff6e41d9d07 - <unknown>
  45:     0x7ff6e4a12484 - <unknown>
  46:     0x7ffbfb866fd4 - BaseThreadInitThunk
  47:     0x7ffbfd0fcec1 - RtlUserThreadStart
 2020-09-12T19:16:28.067Z ERROR window::os::windows::window         > caught Any

When pre-starting wezterm, as mentioned in the workaround, that is running wezterm locally in direct login on Machine 2, tiling wezterm, and then from Machine1 trying RDP connect, disconnect, reconnect, no error log messages appear beyond the initial opengl init, and wezterm silently survives the reconnect.

I ran the the westerm exe, timing it with date commands. After disconnecting, I gave a 2 minute pause, in order to determine whether the crash happens at disconnect, reconnect/local-login. After reconnecting, I immediately, manually, gave the 3rd date command after RDP reconnection. The below log confirms that the crash happens at disconnect.

PS C:\Users\gana> date ; Start-Process "D:\scoop\apps\wezterm\20200909-002054-4c9af461\wezterm.exe" -NoNewWindow -Wait  ; date

Saturday, September 12, 2020 5:43:01 PM
 2020-09-12T21:43:01.954Z ERROR window::os::windows::wgl > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
thread 'main' panicked at 'failed to advise of resize: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer', src\frontend\gui\termwindow.rs:1839:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 2020-09-12T21:44:12.704Z ERROR window::os::windows::window > caught Any
Saturday, September 12, 2020 5:44:18 PM

PS C:\Users\gana> date

Saturday, September 12, 2020 5:46:56 PM

WGL_INFO

running wglinfo64.exe inside RDP session on Machine2 wglinfo_remote_in_RDP.txt running wglinfo64.exe on direct login on Machine2. Seems like no difference wglinfo_machine2_directlogin.txt wglinfo on machine 1 same as in bug #235 Possibly, depending on how RDP works, opengl on machine-1 may not matter to opengl process on machine-2.

Additional context

Add any other context about the problem here.

Misc

One wonders how wezterm might behave under other non-microsoft RDP clients. On Linux, Linux RDP clients, Remmina and Vinagre, allow more control over the RDP session inside the client-window. For example, the remote machine's RDP-session display-size is preconfigurable.

Workaround

Rather that starting wezterm inside the RDP-session, login locally on Machine2 and pre-start wezterm before remote-RDP-ing into it. Started that way, wezterm is resilient to the RDP disconnection,

wez commented 4 years ago

Thanks for the detailed information!

RDP does something "funny" with GPU availability; back in https://github.com/wez/wezterm/issues/40 I added some logic to fall back to the software renderer when the app is opened in an RDP session. Based on the output you shared, I think the more recent changes to WGL/EGL initialization may now have made it possible to get a working OpenGL context in that environment.

The panic message sounds like the RDP OpenGL context may not have enough VRAM to support the screen size at disconnect; I'm assuming that the disconnect results in an excessively large screen size (perhaps due to it being tiled/maximized?) that isn't supportable in that environment. However, if the app is started with access to the true display it has access to the full VRAM available and that might explain why the app survives in that case.

I've pushed 3484478ba261191dc69376f5945d378c031d806d that attempts to recover from a failed attempt to allocate vertex buffers. It should log the old and new dimensions to help us understand a bit better what is going on. I'm not sure if the recovery attempt will prevent a later panic, but I'm hoping that we can at least learn a bit more, and if we're lucky, it will keep things running until you next attach to the session and trigger a resize where it will succeed.

I'd appreciate it if you could try out the windows build associated with that commit once the CI has finished; it should be downloadable from here https://github.com/wez/wezterm/actions/runs/252069384 within ~30 minutes of this comment being posted.

If it is still crashy and getting in your way, a workaround would be to set this in the wezterm.lua file on the remote machine:

return {
  front_end = "Software",
}

that will set the renderer to the Software renderer unconditionally; it will have degraded performance and different visual characteristics that you'll remember from https://github.com/wez/wezterm/issues/235, but should at least keep running.

hgkamath commented 4 years ago
PS C:\Users\gana> $env:RUST_BACKTRACE="full"
PS C:\Users\gana> $env:RUST_LOG="info"
PS C:\Users\gana> Start-Process "D:\Prog\wezterm\windows\WezTerm-windows-20200909-002054-4c9af461-9-g3484478b\wezterm.exe" -ArgumentList @("-V")  -NoNewWindow -Wait
wezterm 20200909-002054-4c9af461-9-g3484478b
PS C:\Users\gana> Start-Process "D:\Prog\wezterm\windows\WezTerm-windows-20200909-002054-4c9af461-9-g3484478b\wezterm.exe" -NoNewWindow -Wait
...
}
 2020-09-13T12:42:06.619Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x22c, handle_type: Unknown } }
 2020-09-13T12:42:06.632Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-13T12:42:06.687Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-13T12:42:06.715Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-13T12:42:06.727Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false
 2020-09-13T12:43:01.283Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 958, pixel_height: 1008, dpi: 96 } -> Dimensions { pixel_width: 958, pixel_height: 1048, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer
 2020-09-13T12:43:16.573Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 958, pixel_height: 1008, dpi: 96 } -> Dimensions { pixel_width: 958, pixel_height: 1008, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer
PS C:\Users\gana>

I usually place my working wezterm window right tiled. 958 = 1080/2-window decorations 1048 = 1080 -status bar which is on top. The dimensions match screensize of Machine-1 where the RDP client is run.

I can run 2 independent wezterm processes as two separate windows So, I run one wezterm as a pre-started wezterm, this is stable to disconnect and does not interfere with my work The other wezterm I can start inside the RDP-session and test the crash on disconnect.

I may need to update the oringial description to update the original issue description to include the regular-resizable-window mode as also susceptible to this bug, I did not notice it earlier as it was frozen for a while, and maybe needs some keyboard typing attempts in the shell before it crashes. I first noticed this when I attempted the below.

I resized the normal window to as large as a right tile (but not really right tiled) and the crash happened

}
 2020-09-13T13:36:58.272Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x25c, handle_type: Unknown } }
 2020-09-13T13:36:58.289Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-13T13:36:58.467Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-13T13:36:58.506Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-13T13:36:58.527Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false
 2020-09-13T13:37:48.663Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 955, pixel_height: 1008, dpi: 96 } -> Dimensions { pixel_width: 955, pixel_height: 1048, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer
 2020-09-13T13:37:59.963Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 955, pixel_height: 1008, dpi: 96 } -> Dimensions { pixel_width: 955, pixel_height: 1008, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer

This below log is with the small regular-sized window as soon as it pops up, not even moving the window/replacing the window. It remained on screen for a while. and I could even alt-tab to it and to the firefox browser in which I type this. I could even move the frozen window around. When it eventually did disapear is didn't log much, it crashed silently, no backtrace

}
opts: Opt {
    skip_config: false,
    cmd: None,
}
 2020-09-13T13:51:07.768Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x20c, handle_type: Unknown } }
 2020-09-13T13:51:07.785Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-13T13:51:07.842Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-13T13:51:07.868Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-13T13:51:07.879Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false

did many reattempts with various resizing/placements. pixel_width: 640, pixel_height: 450, dpi: 96 -- frozen for 40 sec, terminates silently some other unknown sizes -- frozen for some time, terminates silently pixel_width: 640, pixel_height: 1008, dpi: 96 -- frozen for 2 min, then caught "Not enough memory to create the buffer"

attempted to resize frozen window, error handling caught some intermediate window sizes. Expanded window has undrawn area extensions painted black.

}
 2020-09-13T15:02:16.569Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x260, handle_type: Unknown } }
 2020-09-13T15:02:16.592Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-13T15:02:16.746Z ERROR window::os::windows::wgl           > failed to created extended OpenGL context (CreateContextAttribsARB failed), fall back to basic
 2020-09-13T15:02:16.782Z INFO  wezterm::frontend::gui::renderstate > compiling a prog with version 330
 2020-09-13T15:02:16.795Z INFO  wezterm::frontend::gui::termwindow  > OpenGL initialized! Intel(R) HD Graphics 4400 4.3.0 - Build 20.19.15.4568 is_context_loss_possible=false
 2020-09-13T15:02:48.377Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 763, pixel_height: 596, dpi: 96 } -> Dimensions { pixel_width: 773, pixel_height: 611, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer
 2020-09-13T15:02:48.426Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 763, pixel_height: 596, dpi: 96 } -> Dimensions { pixel_width: 785, pixel_height: 626, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer
 2020-09-13T15:02:48.542Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 763, pixel_height: 596, dpi: 96 } -> Dimensions { pixel_width: 809, pixel_height: 645, dpi: 96 }: Error while creating the vertex buffer

Caused by:
    Not enough memory to create the buffer
 2020-09-13T15:02:48.717Z ERROR wezterm::frontend::gui::termwindow  > failed to advise of resize from Dimensions { pixel_width: 763, pixel_height: 596, dpi: 96 } -> Dimensions { pixel_width: 810, pixel_height: 645, dpi: 96 }: Error while creating the vertex buffer

Conclusion If windows is in regular-resizable window mode

fail to see any pattern/correlation between sizes

Even in wezterm 20200909-002054-4c9af461 the regular-resizable window survives a while in frozen state

Freezing and persistence of frozen window could be because of error trapping but insufficient error handling and recovery.

wez commented 4 years ago

Regarding the failure even with the new build; I suspect that there might not be much that can be done to detect and handle this, as the CreateContextAttribsARB call that allows that sort of thing is not available based on your logs. When I spoke with a friend that worked in the game industry about recovering from this sort of thing with opengl, he laughed at me :-p What I think is happening is that the RDP disconnect is leaving the opengl context in an unrecoverable state so it is just a matter of time before the application fails :-/

This thread: https://social.technet.microsoft.com/Forums/windowsserver/en-US/c8295ef8-3711-4576-9293-2c4965280165/opengl-and-remote-desktop suggests that running: tscon 1 /v dest:console can influence how well opengl works in an RDP session (see bottom of that page for a couple of variations on that). Have you tried experimenting with whether your RDP client connects to the console or not? (Another variation on this here: https://stackoverflow.com/a/45723167/149111)

Another article I found was this one that discusses a Group Policy setting that affects GPU availability: https://community.esri.com/thread/225251-enabling-gpu-rendering-on-windows-server-2016-windows-10-rdp It might be interesting to try with and without this enabled to see if it helps.

hgkamath commented 4 years ago

I agree, it may be beyond an opengl client to push through an opengl context in a state of tear-down.

As the RDP use case is very common,

The tscon idea works: Essentially, it disconnects the RDP session in a script, which soon after runs the command outside of the RDP session. So it definitely is a handy way to try run the app outside of the RDP session, when one is unable to start the app locally before attempting the RDP session qwinsta is a tool to query windows session terminal applications.

PS C:\Users\gana> qwinsta
 SESSIONNAME       USERNAME                 ID  STATE   TYPE        DEVICE
 services                                    0  Disc
>rdp-tcp#121       gana                      1  Active
 console                                     3  Conn
 rdp-tcp                                 65536  Listen
PS C:\Users\gana> Start-Process  tscon -ArgumentList @("1", "/dest:console", '/PASSWORD:***') -Wait -Verb RunAs  ; sleep 2 ; date ; Start-Process "D:\Prog\wezterm\windows\WezTerm-windows-20200909-002054-4c9af461-9-g3484478b\wezterm.exe" -NoNewWindow -Wait ; sleep 2 ; date

In one trial, I ran a script that sleeps 5 seconds and runs wezterm, I disconnect RDP soon after starting the script and wait unconnected long enough for the script to have started wezterm and then RDP reconnect. It turns out if done that way, wezterm would start but use the software-renderer as though it doesn't have any GL hardware.

PS C:\Users\gana> sleep 5; Start-Process "D:\Prog\wezterm\windows\WezTerm-windows-20200909-002054-4c9af461-9-g3484478b\wezterm.exe" -NoNewWindow -Wait ; sleep 5 ; date
...
}
 2020-09-13T17:36:53.675Z INFO  wezterm::mux::domain > spawned: WinChild { proc: OwnedHandle { handle: 0x218, handle_type: Unknown } }
 2020-09-13T17:36:53.751Z INFO  wezterm::frontend::gui::termwindow > TermWindow::new_window called with mux_window_id 0 PtySize { rows: 24, cols: 80, pixel_width: 640, pixel_height: 432 } Dimensions { pixel_width: 640, pixel_height: 450, dpi: 96 }
 2020-09-13T17:36:53.934Z ERROR wezterm::frontend::gui::termwindow > OpenGL init failed: The OpenGL implementation is too old to work with glium

I attempted the grouppolicy "Use the hardware default graphics adapter for all Remote Desktop Services sessions”" to see if that allows wezterm to be stable on RDP disconnect, I applied that setting and rebooted. After rebooting, the connection attempt for a fresh RDP session, failed with the following error. screenshot "Your Remote Desktop Services session has ended, possibly for one of the following reasons: The administrator has ended the session, an error occurred while the connection was being established, a network problem occurred." For the time-being, I did not want to complicate the machine's settings so I undid the setting and did not troubleshoot what else I need to enable to get it working. Perhaps needs more prior research as to how many and what changes exactly to get RDP working that way. RDP worked on reboot after reverting gpedit setting.

... reading throught the links .. will update here as I need to note something

The fact that a process started within a RDP session has different resources and privileges is very unsettling. It now makes me think if an app started that way, then it is stunted in some way. Like take for example, firefox/any browser, which uses opengl, started in RDP, will it crash, fail or run underpowered when it is later transitioned to a direct-display later. Maybe, apps that don't need full-hardware graphics, should not try to use api-s more than capabilities of an embedded subset or a vGPU subset.

It seems like some limitation in the design of RDP. The Windows-10 machine is not a simultaneous multi-user graphics-capable system. If a local user switches to a different login, the entire display is switched to that user. If an RDP user connects, the local access is locked with a login-screen. If any user attempts a local login, the RDP session will be disconnected. The RDP user is not given full hardware graphics capabilities to the RDP session. I wonder how this is different from two users who are locally logged in using the switch-user feature and both run their own opengl apps.

Can't wezterm discover the vGPU-state-changes/RDP-disconnect, sleep/wait till the new vGPU-state settles and rejuvenate the front-end ?

Assuming the long term goal of wezterm was to be a new kind of tmux, with opengl accelerated front-end clients, then, thinking along that line, what if

Is it possible to do this yet ?

prabirshrestha commented 3 years ago

Now that I have been using wezterm as default terminal windows due to split support, this issue seems to be a major concern primarily due to work from home and I have to always use RDP to access work machine.

I tried alacritty and seems to have the same crash. Windows Terminal doesn't have this issue. Does this mean using DirectWrite is better on Windows compared to opengl? GVim on windows also uses DirectWrite and doesn't have crashes.

wez commented 3 years ago

Assuming the long term goal of wezterm was to be a new kind of tmux, with opengl accelerated front-end clients, then, thinking along that line, what if

* one just starts a sort of wezterm mux server process
* connects to it with wezterm clients
* clients will get whatever best opengl context they have
* if clients get killed/frozen/terminated, a fresh new frontend can be started, and then made to connect to the wezterm mux server and then reopen tabs to preexisting terminals, bringing back the state/text/running shells they had in them.

Is it possible to do this yet ?

Yes, this is described here in the docs here: http://wezfurlong.org/wezterm/multiplexing.html#unix-domains

wez commented 3 years ago

I picked OpenGL because it promises to enable GPU acceleration for multiple platforms with basically the same API for all of them, making it easier to support. I do find it frustrating that there are very sharp edges on Windows; this issue is one, and #156 is another that I find particularly troublesome, because there isn't a way to handle and recover from this situation.

I've been thinking a bit about possible solutions to this, including targeting DirectWrite/2D/3D. My line of thinking is this:

Which leads me to conclude that a decent sounding state would be:

There doesn't appear to be a ready-to-download binary distribution of Mesa on Windows, so some amount of effort will need to go into that; I'm not sure how long it takes to build and whether it is feasible to check it out and build it as part of the CI, or whether I'll need to build and upload those binaries somewhere. As I mentioned: I'm not often in dev mode at the same time as being on Windows, so investigating this aspect has a really high activation cost.

wez commented 3 years ago

http://mesa.fdossena.com/ has a pre-built opengl32.dll: https://downloads.fdossena.com/geth.php?r=mesa64-latest Putting that dll in the wezterm.exe directory will cause wezterm to use the llvmpipe renderer. I've been trying to deploy that under a different name and load it dynamically; while the code thinks that is successful, the window just renders in white. AFAICT, there are special assumptions about the name of the dll which makes automatic fallback problematic.

There's some relevant portions of code here: https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/src/gallium/frontends/wgl/stw_ext_context.c#L142 https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/src/gallium/frontends/wgl/stw_wgl.c#L70 https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/src/gallium/frontends/wgl/stw_wgl.c#L386-398

hgkamath commented 3 years ago

... saw your above comment, submitted 7 min before I submitted this.

I loathe to go down the WSL, WSL2 requires Hyper-V, WSL2 takes a lot of space, tricky to get choice of distribution. but I will take look at it.

I installed a fuller fedora with GUi in the vbox vm and noticed that running wezterm in linux defaults to the scrawny looking inbuilt software renderer, as the VM does not have true 3D hardware display card.

In linux, it is possible to choose renderer via environment variables. Mesa environment variables Seems like the fact that there is no default or fallback is intentional.

[gana@localhost ~]$ glxinfo | grep "renderer "
Extended renderer info (GLX_MESA_query_renderer):
OpenGL renderer string: SVGA3D; build: RELEASE;  LLVM;

[gana@localhost ~]$ export LIBGL_ALWAYS_SOFTWARE=true

[gana@localhost ~]$ export GALLIUM_DRIVER=swr
[gana@localhost ~]$ wezterm
[gana@localhost ~]$ # fails to create DRI screen, fallback to inbuilt renderer
[gana@localhost ~]$ glxinfo | grep "renderer "
libGL error: failed to create dri screen
libGL error: failed to load driver: swrast
X Error of failed request:  GLXBadContext
  Major opcode of failed request:  150 (GLX)
  Minor opcode of failed request:  6 (X_GLXIsDirect)
  Serial number of failed request:  48
  Current serial number in output stream:  47

[gana@localhost ~]$ export GALLIUM_DRIVER=softpipe
[gana@localhost ~]$ wezterm
[gana@localhost ~]$ # nice looking wezterm
[gana@localhost ~]$ glxinfo | grep "renderer "
Extended renderer info (GLX_MESA_query_renderer):
OpenGL renderer string: softpipe

[gana@localhost ~]$ export GALLIUM_DRIVER=llvmpipe
[gana@localhost ~]$ wezterm
[gana@localhost ~]$ # nice looking wezterm
[gana@localhost ~]$ glxinfo | grep "renderer "
Extended renderer info (GLX_MESA_query_renderer):
OpenGL renderer string: llvmpipe (LLVM 11.0.0, 256 bits)

Similarly, in windows, dropping a opengl32.dll is one way a binary is made to use a custom opengl renderer. ex: fdossena

Bundling/depending Mesa does make wezterm installer bigger.

prabirshrestha commented 3 years ago

front_end = 'Software' seems to work. I had to search the code to see the key. Would be good if this was documented. I was initially trying frontend = 'Software'.

Having multiple backend support is definitely not good and incurs huge cost.

Not sure what exactly Neovide does but this release seems to work for RDP https://github.com/Kethku/neovide/releases/tag/0.3.0 . Latest one crashes. It uses vulkan but doesn't for everyone I guess. Other option would be to have vulkan and pure software render only.

I have absolutely no idea about graphic system but would libraries such as SDL2 or gfx-rs help? Specially with this bold statement from gfx-rs.

gfx-rs makes low-level GPU programming portable with low overhead. It’s a single Vulkan-like Rust API with multiple backends that implement it: Direct3D 12/11, Metal, Vulkan, and even OpenGL.

If there is a hello world app more than happy to build my self and try it out if it works in RDP or not.

wez commented 3 years ago

would libraries such as SDL2 or gfx-rs help?

SDL2 was one of the first I tried; it has a pretty rough user experience on Linux, making the whole screen flicker each time a window is opened.

All of the other GUI related libraries in the Rust ecosystem are based on winit which traditionally had huge problems even with opening a second window and which also have relatively poor performance on Linux.

I think OpenGL is probably the best abstraction we have available and I think the llvmpipe software renderer is fine as a fallback, provided that we can nail automatically activating it.

https://docs.microsoft.com/en-us/windows/win32/termserv/detecting-the-terminal-services-environment has details on how to programmatically detect an RDP session; there's quite a lot of code involved and I didn't get around to porting it to rust.

Rather than looking at another gfx backend, if you feel like writing a bit of rust code, I'd appreciate it if someone were to write a rust implementation of the IsCurrentSessionRemoteable from that article and then glued it into this bit of code and submit a PR :-)

https://github.com/wez/wezterm/blob/master/src/frontend/gui/mod.rs#L57

        #[cfg(windows)]
        {
            if !is_running_in_rdp_session() {
                // Using OpenGL in RDP is not safe/reliable on disconnect,
                // so fallback to software rendering
                ::window::prefer_swrast();
            }
        }
wez commented 3 years ago

I just pushed 9e46ac83be8284c5cc5a8bf4232f248b44af2f4f to for software mode in an RDP session. I've test this in !RDP but didn't have a chance to try it in an RDP session before starting my working day. With this change you shouldn't need to specify the front end in your configuration any longer; if you could try this out and let me know how well it works, I'd appreciate it!

prabirshrestha commented 3 years ago

tested in rdp. it works. i can now remove software render from wezterm.lua

wez commented 3 years ago

Great, let's call this done!

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.