search

mobile-shell / mosh

Mobile Shell
https://mosh.org
GNU General Public License v3.0
12.65k stars 736 forks source link

Issues observed with PowerShell on Server-Side #1258

Open heywoodlh opened 1 year ago

heywoodlh commented 1 year ago

I use PowerShell Core (currently 7.3.2) on my servers (NixOS, Arch Linux) running OpenSSH version OpenSSH_9.2p1, OpenSSL 3.0.8 7 Feb 2023.

When I use mosh over ssh with PowerShell I observe two things I don't encounter with a normal ssh client:

image

Is there any way to get mosh to play more nicely with PowerShell (or vice versa) -- I'd love to have these issues resolved. I am not sure what further information to provide, so please let me know what additional information would be useful in diagnosing the exact issue.

achernya commented 1 year ago

What terminal emulator are you using? In theory the specific shell should not matter, but I admit I don't know how powershell interacts with its terminal emulator.

heywoodlh commented 1 year ago

@achernya I have observed the issues I described while using mosh in the following terminal emulators:

achernya commented 1 year ago

Do you have screenshots from, say, xterm, rxvt-unicode, gnome-terminal, or Terminal.app? Preferably showing side-by-side of ssh and mosh? Those are the terminals I am most familiar with. I'd normally ask for the output of echo $TERM ... but I have no idea if powershell respects $TERM. Also do the artifacts go away if you run powershell under screen/tmux?

eminence commented 1 year ago

Also, what version of mosh, on both the client and server?

heywoodlh commented 1 year ago

So I just found that adding --no-ssh-pty seemingly resolves the issue. Is there a way to make that a default in my PowerShell environment, i.e. with an environment variable (worst case scenario, I could just create a PowerShell alias for mosh --no-ssh-pty)?

Do you have screenshots from, say, xterm, rxvt-unicode, gnome-terminal, or Terminal.app? Preferably showing side-by-side of ssh and mosh?

If you'd like me to, I can send these screenshots but with figuring out the --no-ssh-pty flag resolves this issue for me, I'm guessing that it isn't a terminal emulator specific issue?

what version of mosh, on both the client and server?

1.4.0 (on both)

Also do the artifacts go away if you run powershell under screen/tmux

Funny enough, I use PowerShell in tmux, and it does not go away with it (not counting my recent discovery of --no-ssh-pty).

eminence commented 1 year ago

I haven't been able to reproduce this, using kitty v0.21.2 and PowerShell Core 7.3.2 on Debian. I don't have PowerShell yet as my default shell, so I tried to reproduce with:

mosh me@server.com pwsh

I don't really understand why no-ssh-pty has an impact here. I'd be interested in capturing what powershell is sending to your client. Can you capture that? Perhaps with something like:

mosh you@server.com -- script /tmp/pwsh.script -c pwsh

(Hopefully the same stuff gets injected to your client). Then exit powershell and send us the pwsh.script file

heywoodlh commented 1 year ago

Attached is pwsh.script (had to rename it to .txt to upload it to GitHub) pwsh.txt

heywoodlh commented 1 year ago

To me, it looked like the same stuff was injected after script ran

achernya commented 1 year ago

Interestingly enough the script recording shows that ^[[48;1R^[[48;1RPowerShell 7.3.2 is verbatim a caret and a open square bracket, not a \[ escape sequence. This feels like a powershell bug to me, because if I replace it with an escape sequence character it's correct.

heywoodlh commented 1 year ago

This feels like a powershell bug to me

Yeah, that would make sense to me. I was thinking it could be my PowerShell profile, but upon logging into a server with PowerShell Core as it's default shell without any of my PowerShell profile stuff setup (so just using the default PowerShell profile), I seem to experience the same issue.

In the meantime, as a workaround I have created a PowerShell function to inject --no-ssh-pty into my mosh commands: https://github.com/heywoodlh/conf/blob/1d1e4b32d10983ddfafb3358b7a58a40bbf2c159/dotfiles/config/powershell/not-windows.ps1#L33-L37

If there's anymore information I can provide to help drill into this further please let me know and I'd be happy to provide it. I'm not too sure how to move forward with this on my end.

eminence commented 1 year ago

I guess it could also be something that runs before PowerShell, but if you don't see these weird characters with another shell (say, bash), then that does kind of point toward PowerShell.

I'm a bit confused as to what ^[[48;1R actually is. I can't find any CSI functions that end in R (except for XTREPORTCOLORS but that doesn't have the same form). I do notice that your original screenshot had ^[[21;1R and your latest script has ^[[48;1R -- I wonder if is supposed to be a DECSTBM, which ends with a lower-case r (do the numbers change based on your terminal size @heywoodlh ?). Even if so, this would be a weird bug, and doesn't explain why we're seeing a verbatim ^[[ instead of CSI

Anyway, I recognize that you may no longer want to spend time continuing to debug an issue for which you have a workaround. I would be curious to know if you see this issue with bash or how your terminal size might impact the output, if you don't mind.

heywoodlh commented 1 year ago

I would be curious to know if you see this issue with bash or how your terminal size might impact the output, if you don't mind.

Nothing on my client side seems to contribute to this. I only see this when PowerShell is my default shell on the server-side. When PowerShell is not my default shell, I do not experience this issue.

I recognize that you may no longer want to spend time continuing to debug an issue for which you have a workaround

I definitely would love to help find a fix to this rather than having to workaround this, so keep the suggestions coming. :)

eminence commented 1 year ago

How about getting an strace dump to confirm (or refute) the theory that it's pwsh that's printing this stuff?

eminence commented 1 year ago

also, you said specifically that you see this when PowerShell is your default shell:

heywoodlh commented 1 year ago

So here's how it looks when I login with mosh to a server which has BASH as its default shell (no weird symbols injected, looks normal):

image

What if PowerShell is your default shell, but you use bash instead?

image

Same issues as I described:

To test this, I ran this command (let me know if there is a better way): 

mosh <host> bash

What if bash is your default shell, but you use PowerShell instead?

No issues:

image

To test this, I ran this command (let me know if there is a better way): 

mosh <host> pwsh

How about getting an strace dump to confirm (or refute) the theory that it's pwsh that's printing this stuff?

Can you provide an example command? I'm not sure if you want strace on the server-side or client-side (my suspicion is that it's something with PowerShell Core + SSH on the server side).

eminence commented 1 year ago

ok, this is even more mysterious. If we see the weird stuff even when powershell isn't used, then maybe it's not powershell's fault at all.

I think we need an strace on the server, to see what's printing this (assuming that someone on the server-side is printing it).

So something like:

mosh <host> -- strace -ff -o /tmp/strace.output --string-limit=128 bash

Then send all the strace output files. What I'm looking to do here is identify exactly what process is writing the ^[[48;1R^[[48;1R string

eminence commented 1 year ago

@heywoodlh Any news on this?

heywoodlh commented 1 year ago

@eminence sorry for the delay, got distracted with life 😄

Just FYI, this is me mosh-ing into a NixOS server, which is why the file paths for binaries and such might be a bit odd.

strace.output.849741.txt strace.output.849740.txt strace.output.849739.txt strace.output.849670.txt

omaidb commented 5 months ago

Did you use Windows server? How did you install mosh on Windows server? Can you guide me to install mosh on Windows server?