SSH causes intermittent issues

[taman0753]

Discord username (optional)

No response

Describe the bug

When i do SSH into some sever this happens frequently .

channel 21: open failed: connect failed: open failed channel 21: open failed: connect failed: open failed channel 23: open failed: connect failed: open failed channel 21: open failed: connect failed: open failed channel 23: open failed: connect failed: open failed channel 25: open failed: connect failed: open failed channel 21: open failed: connect failed: open failed channel 23: open failed: connect failed: open failed channel 25: open failed: connect failed: open failed channel 21: open failed: connect failed: open failed channel 23: open failed: connect failed: open failed channel 25: open failed: connect failed: open failed channel 21: open failed: connect failed: open failed channel 21: open failed: connect failed: open failed channel 23: open failed: connect failed: open failed

To Reproduce

After doing ssh to machine.

Expected behaviour

No response

Screenshots

No response

Operating System

MacOS

OS Version

12.6

Shell Version

version 3.2.57

Warp Version

v0.2022.10.11.08.13.stable_01

Additional context

No response

Does this block you from using Warp daily?

Yes

Warp Internal (ignore): linear-label:b8107fdf-ba31-488d-b103-d271c89cac3e

No response

[szgupta]

Hey @horacimacias, sorry to hear the problem is still occurring. I think in this case, it might help to bump up MaxSessions on the remote host. The default value is 10. Do you have access to edit the sshd config on the host?

[horacimacias]

thanks, I can definitely give this a try and report back. Is it "normal", though, for just 1 ssh connection and running ls to need more than 10 sessions?

[szgupta]

That's a great question. So one connection is needed for the interactive shell itself. The other connections are used to run commands for completions, syntax-highlighting, etc.

We definitely want to eventually limit how many commands we run at once based on the host's ssh config (specifically the MaxSessions value). That's something we'll look into for the future

[taman0753]

@szgupta @alokedesai Still facing same issue , moving to different terminal for now. Happening for mostly all remote machines.

[szgupta]

Hey @taman0753, sorry to hear the problem is still occurring. The fix we released is definitely a mitigation and we're still actively investigating a total fix.

Out of curiosity, on these remote machines, what is the MaxSessions value in the server's sshd_config?

[taman0753]

@szgupta Not specified explicitly any value.

[szgupta]

@taman0753 reached out over email to debug further!

[lgebing]

Just jumping in here to say that after a period of not encountering this issue after the v0.2022.11.29.08.03.stable_07 release it is now starting to pop up again.

My Setup seems to pretty much exactly match the one mentioned here #2422 (though I didn't compare the sshd_config values).

[szgupta]

@lgebing could you confirm what MaxSessions is set to for that machine's sshd_config?

[lgebing]

In the config file it is commented out: #MaxSessions 10

After running sshd -T I got maxsessions 10, so the default value. Most settings should be default.

[garythung]

I'm seeing this on the latest version, v0.2022.12.13.08.04.stable_01.

sshd_config has MaxSessions 10

Warp features ON: ssh wrapper, syntax highlighting for commands Warp features OFF: open completions menu as you type

[taman0753]

@szgupta @alokedesai Any updates here

[taman0753]

@dannyneira Still I am getting this issue

[dannyneira]

@taman0753 I suggest disabling the SSH Wrapper for now as a temp workaround, it will disable Warp features on ssh sessions, but the errors should no longer show up. I believe we have some SSH improvements being worked on, just no official ETA. Thanks for your patience, we'll post updates on this thread.

[taman0753]

@dannyneira Yes thats a workaround , but after disabling it I am not able to use auto-completions in a SSH box

[zhengxiexie]

Hi, I'm troubled by this problem for several months, how long will it be fixed, Thanks~~

[vorporeal]

Hey all, I'm actively looking into this and have a couple leads on fixes/mitigations. Hoping to have something in next week's build. I might also reach out here to see if anyone is willing to try out a build with a possible fix to see if it actually helps.

[dannyneira]

Hey folks we're aware of the issue and working on a fix. We'll post any updates on this thread.

[vorporeal]

Update: unfortunately the fix I was working on isn't going to pan out (it can cause other, worse issues). We're still looking into ways to fix this - we have an engineer who is working on building an alternative method of generating completions results which would eliminate this entirely.

[vorporeal]

Ok another update: I have a change that won't totally fix the issue, but should reduce the frequency substantially, and should produce less spam when these messages do occur. Should go live in next week's release.

[flyerhawk]

Is there an update on this? It's getting worse for me to the point I may need to move off of this product.

[vorporeal]

Hey @flyerhawk, sorry to hear that - we're working on fixing this but it's somewhat involved. For the time being, one option would be to disable the SSH wrapper in Settings > Features to return to a typical (non-Warpified) SSH experience, which will also eliminate the errors.

[rvalenzuelar]

Hi guys, I am using warp v0.2023.03.07.08.02.stable_01 with and without the SSH wrapper activated and seeing the same message repeated several times after cd to a directory. I am on a macOS Monterey machine.

[dannyneira]

Sorry to hear about this @rvalenzuelar, Do you see the message only on certain machines ( remote OS version ) or any ssh session? Mind including a screenshot of the error message you see along with the remote ssh session OS and shell type and version? ( make sure to redact any IP or username )

[taman0753]

@dannyneira When can we expect a stable build for this ?

[dannyneira]

Hey @flyerhawk, sorry to hear that - we're working on fixing this but it's somewhat involved. For the time being, one option would be to disable the SSH wrapper in Settings > Features to return to a typical (non-Warpified) SSH experience, which will also eliminate the errors.

@taman0753 We're working on it, but it's a tricky one so we appreciate your patience. Please see the workaround in the quoted comment above. We'll post more updates if/when we have them on this thread.

[lgebing]

Hey, just got the following message after running a command that would sometimes lead to the bug:

mm_receive_fd: recvmsg: expected received 1 got 0
mux_master_process_new_session: failed to receive fd 1 from client

Maybe this is helpful.

[taman0753]

@dannyneira @alokedesai Any quick update here ?

[alokedesai]

Hey @taman0753, we have an engineer who is actively working on a fix for this. Our current estimate for when this will launch is mid-May but I'll followup as soon as I have a more concrete launch date.

[forrestfwilliams]

Experiencing the same issue.

[hazel-ys-lin]

Experiencing the same issue (Especially when running remote SSH to AWS EC2 instances (ubuntu 22 now), not happened yet in my local machine (MacOS Ventura)

[p0we7]

I am using v0.2023.06.08.02.stable_00 with the default configuration. When I connect to the remote SSH and press Tab to complete, I get the following error.

channel 5: open failed: administratively prohibited: open failed
channel 17: open failed: administratively prohibited: open failed
channel 9: open failed: administratively prohibited: open failed
channel 18: open failed: administratively prohibited: open failed
channel 8: open failed: administratively prohibited: open failed
channel 11: open failed: administratively prohibited: open failed
channel 18: open failed: administratively prohibited: open failed

[taman0753]

@alokedesai @dannyneira Any updates on this bug , its been long pending , it has made me move out of Warp.

[zachbai]

Hey all, sorry for the silence here. The issue here has to do with the implementation of completions and syntax/error highlighting for SSH sessions.

My short-term suggested workaround for this (at the moment) is using our Subshells feature, which you can use for regular SSH sessions with the following steps:

• (one-time setup) Add "command ssh .*" to your list of "subshell commands" (see "Configuring subshell-compatible commands" docs here)
• Prefix ssh with command, e.g. instead of ssh me@somehost, command ssh me@somehost. This circumvents the SSH wrapper and will trigger the subshell logic instead.

Subshells uses a different implementation for completions/syntax highlighting that shouldn't trigger the errors you're seeing above.

Longer-term, I'm investigating deprecating the existing (clearly buggy) completions implementation for SSH and replacing it with the implementation used today by subshells. There's no firm timeline for this but the workaround I mentioned above should get you 80% of the way there for now.

Thanks for your patience and do follow up if you encounter issues with this!

[saksham2410]

Any update on this issue? Facing it a lot lately.

[xpader]

This happends to me when I terminate a long cat by Ctrl + C sometimes.

[seththorup]

We have this issue when ssh into older fedora 27 systems. Also interested in a fix as the issue seems to have just started with that latest updates in warp. the work around mentioned below is a good work around.

Hey all, sorry for the silence here. The issue here has to do with the implementation of completions and syntax/error highlighting for SSH sessions.

My short-term suggested workaround for this (at the moment) is using our Subshells feature, which you can use for regular SSH sessions with the following steps:

• (one-time setup) Add "command ssh .*" to your list of "subshell commands" (see "Configuring subshell-compatible commands" docs here)
• Prefix ssh with command, e.g. instead of ssh me@somehost, command ssh me@somehost. This circumvents the SSH wrapper and will trigger the subshell logic instead.

Subshells uses a different implementation for completions/syntax highlighting that shouldn't trigger the errors you're seeing above.

Longer-term, I'm investigating deprecating the existing (clearly buggy) completions implementation for SSH and replacing it with the implementation used today by subshells. There's no firm timeline for this but the workaround I mentioned above should get you 80% of the way there for now.

Thanks for your patience and do follow up if you encounter issues with this!

[samiran-kawtikwar]

@taman0753 Just to confirm, you aren't seeing this in other terminals? Does this still repro in Warp if you disable the "Warp SSH Wrapper" setting?

This helped me resolve the issue for noe, but it disables all the warp features. So not really useful :(.

[arpanr]

any update on this?

[SevkiBekir]

I've encountered the problem for a long time. Is there any solution?

[mdhender]

I am using v0.2023.10.03.08.03.stable_01 (would be great if that were copyable on the Settings > Account dialog since it's prominently displayed there and going to the About tab isn't obvious, but I digress).

I have the same problem when I SSH into a VM and am typing commands in a directory with a large number of entries (almost 50k). It is the only place that I've seen this happen.

channel 13: open failed: connect failed: open failed
channel 14: open failed: connect failed: open failed
channel 3: open failed: connect failed: open failed
channel 14: open failed: connect failed: open failed
channel 14: open failed: connect failed: open failed
channel 5: open failed: connect failed: open failed
channel 3: open failed: connect failed: open failed
channel 13: open failed: connect failed: open failed

FWIW, the box at that top of the terminal displays "Seems like your completions are not working..." (and that text isn't selectable, either).

[ghost]

same problem for me for a long time. I encountered this problem only for a specific remote server (HPC server), while other remote servers could work normally. Thus, I do not know if it is related to some settings in the HPC server?

[xyeoda]

having the same issue too...keen to hear if there is going to be some fix for it

[Dedsecproject]

disabling SSH Wrapper did it for me. Though, I can't use warp's features on the next machine I remote in.

[xyeoda]

I think it has something to do with the SSH server on the host. I'm using it in a home lab so I removed ssh server and installed it again plus removed the fingerprint from the client machine. It's working for that host again.

[smxdevst]

any updates on this?

happens on Amazon Linux 2 with updated /etc/ssh/sshd_config MaxSessions = 10

[RafailFridman]

It also happens for me on a Mac (Sonoma 14.2.1) when trying to connect to a server running on Centos 7.0 Is there any way to temporarily suppress these warnings maybe?

[siegero]

Running Warp v0.2024.01.09.08.02.stable_01 on macOS Sonoma 14.2.1. After the last updates of Warp this issue has been getting much worse.

I have quite a few VMs running Debian. Those still on Debian 11 (Bullseye) and bash 5.1 is not as much affected by this as Debian 12 (Bookwork) with bash 5.2.

Tried workaround with 'MaxSessions' in sshd_config with no effect.

[liuhao-97]

I am using v0.2024.02.06.08.02.stable_01 on macOS. Same problem even worse. Other terminal like iterm works well.

[zhangjh]

Sad to see this problem had lasted too long without being solved. Hope it'll solve soon.