Session Manager Lags and Slows Down when Copying/Pasting

[e-moshaya]

When copying and pasting text into a file for example using session manager, there is extreme lag and very slow. Whereas with ssh, pasting text into files is instantaneous with no noticeable lag.

[kellertobias]

+1 for me also. I also recognized that SSM has problems when e.g. crolling through a file in nano. it doesn't stop when I stop scrolling, but seems to send all keystrokes, no matter what.

[dpujadas]

It seems that the first 100 characters in clipboard are sent almost instantaneously but, from there, a considerable lag appears in each new character.

[TomGMan]

This is a thing indeed. Trying to paste in a few lines (100+) of bash script into an instance yesterday, which turned my terminal window into a screensaver (a pretty one, though).

[AmyZhangAmz]

Thank you for the feedback! We will look into this.

[jonatcorus]

This issue is fairly major as it makes it hard to use SSM Sessions as a replacement for SSH to a jumpbox.

[rnhurt]

The slowdown doesn't seem to be consistent though. Sure, after the first 100 characters or so you hit the wall, and for quite a while after that it's pretty slow. But then I've seen it speed back up somewhat after a time. If you are unfortunate enough to cat a really long log file you can see this in action. It's still really, really slow but it does seem to "catch it's breath" after a while and smooth out a bit.

Then again, maybe I'm just wishing for it to be fast. :grimacing:

[M1ke]

Just to add we are experiencing this too. I am attempting to use only SSM but still have SSH available. It would be ideal to handle this as it prevents moving away from SSH otherwise. Obviously there's some work with the protocol (HTTP is different from SSH after all) but there are other fast signalling systems that could maybe achieve higher throughput, such as websockets.

[rnhurt]

@M1ke, it's my understanding that this agent already uses websockets for its communication. If you look in the vendor directory you can see the Gorilla GoLang websocket library and references to the library in the code and tests.

Looking at the Gorilla documentation the buffer size is set to 4096 by default. Perhaps this buffer size is too low for this project and increasing it would eliminate the slowness. :thinking:

[jacekbarecki]

+1

[rnhurt]

I was working with Session Manager quite heavily this week and found that it felt much more snappy. So I thought I would add a bit of detail to this issue. For a simple test I opened a file and copy and pasted some lines to see if/when it would slow down. It turns out that pasting in 50 simple lines shows the problem pretty well. My test show that it got to about line 35 before it slowed down.

Interestingly, viewing the same file didn't slow down at all. In fact, I can cat pretty large files without too much slowdown. It's still not near as fast as an SSH session but it seems to be manageable.

Agent version: 2.3.372.0 Instance: t3.medium running Amazon Linux 2 Command: vi /tmp/count.txt Content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

[wfeng-fsde]

We are seeing the same slowness when pasting in ssm sessions. The first 100 or so appears quickly and then it's like watching grandma typing up an early childhood autobiography...

[donalddewulf]

Same for me, copy pasting is a no go into ssm sessions. Also ctrl-c doesn't seem to be recognized most times. Especially frustrating if you are trying to exit a tail of a big file. I found it easier to close the SSM session and to restart it in those scenarios. Doing ls -al on a big directory sometimes lags the session for a whole minute.

[BenCoughlan15]

any sign of improving this? It's super painful.

[tomotway]

+1

[KostiantynKopytov]

+1

[hogie48]

+1

[JSekhon321]

+1

[mdavani]

+1

[rmlsun]

+1

[kellertobias]

Dafuq. Stop the "+1" spam! It is not helpful and only annoying those who have already commented.

Use the thumbs up on the original post.

[BenCoughlan15]

Dafuq. Stop the "+1" spam! It is not helpful and only annoying those who have already commented.

Use the thumbs up on the original post.

+1

[obazoud]

could someone confirm that the issue comes only from websocket buffer size ?

[alewando]

I experienced this bad performance also, but it got much better when I upgraded to the latest session-manager-plugin (https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html).

It may have been commit 346cb4d3a714c788b3c9ce1d6d99a4d3fd5bcf87 that fixed this

[obazoud]

Thanks @alewando - work better with latest session-manager-plugin.

[dpujadas]

It looks like latest version solved problems with stdout (eg, cat long files), but copy+paste problem still persists (at least for me). This is my user experience (after pasting 10 lines, ~100 chars, things get slow):

In addition, ctrl+c doesn't work. So, if you paste a lot of content, you better go and get some coffee ...

[AKoetsier]

@dpujadas which version of the session-manager-plugin are you using (session-manager-plugin --version)? I just installed version 1.1.17.0 and the problem with slow pasting seems to be gone.

[dpujadas]

@dpujadas which version of the session-manager-plugin are you using (session-manager-plugin --version)? I just installed version 1.1.17.0 and the problem with slow pasting seems to be gone.

Yes, 1.1.17.0 finally solved the problem for me. Does anybody know how to install a specific version of session-manager-plugin? Following the instructions in the documentation, you always get the latest, but with a docker build (which was my case), docker was using the cache as the URL didn't change ...

[sindhup08]

Even with the latest version 1.1.17.0 , if the code is less than 500 lines it is working fine , when the code is more than 650 lines , it just disconnects my connection.

[dorner]

The new version is definitely a bit better. Trying to paste really long lines still sees slowdown though. It's faster than it was, but still incredibly slow compared to real SSH. Try pasting a list of 3000 numbers in a single line and you can see what happens.

[dorner]

Also, if you want to stop the paste because of the slowdown with ctrl-c, it just plain doesn't work. :(

[mattolenik]

Even regular typing has noticeable lag when connecting across regions.

[nitikagoyal87]

This issue has been fixed, please upgrade to the latest versions of Session Manager Plugin and SSM Agent.

[Shumakriss]

@nitikagoyal87 just upgraded to 2.3.1205.0 and I'm experiencing poor data transfer rates. Perhaps I'm not at latest, it's not completely fixed, or I have another problem. Could you specify the minimum version? Thank you!

Edit: The issue is still open as well.

[thomkenn]

Still experiencing this, attempting to copy paste large files is turning my box into a screensaver, and the only way to escape is to refresh the page and lose everything you did

[carcinocron]

if you paste something too large (an SQL query in my case), a small number of random characters will be silently dropped.