In windows can't use arrow navigation exec bash into container

[phazei]

BUG REPORT INFORMATION

docker version: Client version: 1.6.2 Client API version: 1.18 Go version (client): go1.4.2 Git commit (client): 7c8fca2 OS/Arch (client): windows/amd64 Server version: 1.6.2 Server API version: 1.18 Go version (server): go1.4.2 Git commit (server): 7c8fca2 OS/Arch (server): linux/amd64

docker info: Containers: 15 Images: 309 Storage Driver: aufs Root Dir: /mnt/sda1/var/lib/docker/aufs Backing Filesystem: extfs Dirs: 339 Dirperm1 Supported: true Execution Driver: native-0.2 Kernel Version: 4.0.3-boot2docker Operating System: Boot2Docker 1.6.2 (TCL 5.4); master : 4534e65 - Wed May 13 21:24:28 UTC 2015 CPUs: 4 Total Memory: 1.957 GiB Name: boot2docker ID: FKPI:HE56:LNVN:B25Q:RDMA:YB5E:AENE:Y2HR:72C4:CNUS:LNMP:3G7M Debug mode (server): true Debug mode (client): false Fds: 32 Goroutines: 49 System Time: Tue Jun 9 06:00:31 UTC 2015 EventsListeners: 0 Init SHA1: 7f9c6798b022e64f04d2aff8c75cbf38a2779493 Init Path: /usr/local/bin/docker Docker Root Dir: /mnt/sda1/var/lib/docker Username: rasterdocker Registry: [https://index.docker.io/v1/]

uname -a: MINGW64 _NT-6.1 raster-dev-PC 2.1.2(0.288/5/3) 2015-05-25 15:48 x86_64 Msys

Provide additional environment details (AWS, VirtualBox, physical, etc.): Windows 7 using native windows client in either cmd or msys2.

List the steps to reproduce the issue: 1) docker run --rm -it --entrypoint=//bin/bash ubuntu:14.04 2) type anything, press enter, type some more 3) Hit left, right, up, down, home, end.

Describe the results you received: Nothing happens: cursor does not move.

Describe the results you expected: cursor moves left, right, up one in history, down one in history, beginning of line, end of line

Provide additional info you think is important: Tried it on 2 windows 7 machines, same results. It does work if I do boot2docker ssh first and follow same steps.

[phazei]

Updated!

[phemmer]

This is likely a result of an unusual terminal emulator. Try running

docker run -e TERM --rm -it --entrypoint=//bin/bash ubuntu:14.04

[phazei]

TERM doesn't seem to make a difference. Same results with cygwin, linux, xterm-color

[tmds]

I have the same issue when using ConEmu, but not when using the default windows terminal.

[phazei]

I'm also in ComEmu actually, hmm. I tried detaching the cmd tab and running docker exec, it was weird as well, all sorts of funky characters, think that was because of something when it's disconnected it doesn't even have the normal windows console since it's resizable. I ran cmd as admin since it doesn't attach auto attach, and ran docker exec, and the arrow keys seemed to work fine.

[Maximus5]

When you are in ConEmu and arrow navigation doesn't work, reveal RealConsole by Ctrl+Win+Alt+Space, set focus into it and try keys. Result?

[tmds]

RealConsole has the same behavior, i.e.: arrow keys don't work

[Maximus5]

If so, I believe the problem is not in ConEmu or docker itself. But in the configuration of your shell. It starts with different ways I believe.

[tmds]

Summary of findings: When running a command like docker run -ti --rm debian:jessie /bin/bash

• arrow keys are working when started in 'cmd' or 'powershell'
• arrow keys are not working when started in 'ConEmu' via 'cmd' or 'powershell'
• arrow keys are working when started in 'ConEmu' using 'boot2docker ssh' via 'cmd' or 'powershell'

From this I guess that there is some incompatibility in how 'docker run' and 'ConEmu' work with the arrow keys.

[phazei]

Not sure if this might shed some light on it, but if I choose "detach" on the current bash window, then go into a docker container, it gives me strange characters, maybe their particular encoding might explain something?

adam@dev-PC MINGW64 ~
$ docker.exe run --rm -it ubuntu:14.04 //bin/bash -l -i
root@edfa63f0e3e4:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@edfa63f0e3e4:/# exit
logout

# detach from ConEmu here and do exact same thing again:

adam@dev-PC MINGW64 ~
$ docker.exe run --rm -it ubuntu:14.04 //bin/bash -l -i
←]0;root@ad7bbdddd009: /root@ad7bbdddd009:/# ls
←[0m←[01;34mbin←[0m  ←[01;34mboot←[0m  ←[01;34mdev←[0m  ←[01;34metc←[0m  ←[01;34mhome←[0m  ←[01;34mlib←[0m  ←[01;34mlib64←[0m  ←[01;34mmedia←[0m  ←[01;34mmnt←[0m  ←[01;34mopt←[0m  ←[01;34
mproc←[0m  ←[01;34mroot←[0m  ←[01;34mrun←[0m  ←[01;34msbin←[0m  ←[01;34msrv←[0m  ←[01;34msys←[0m  ←[30;42mtmp←[0m  ←[01;34musr←[0m  ←[01;34mvar←[0m
←]0;root@ad7bbdddd009: /root@ad7bbdddd009:/# exit
logout

adam@dev-PC MINGW64 ~
$

EDIT:

stty -a also outputs the same thing either way.

speed 38400 baud; rows 68; columns 240; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 -hupcl -cstopb cread -clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff -iuclc -ixany -imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke

[crmckenzie]

We are so close to awesomesauce on this. Would love to see it addressed. If I knew anything about terminal emulation I'd jump in and help.

[aarontgrogg]

Seems to be working for me, but not for the command the tab automatically ran when it opened. For instance, I have a Grunt watch command set to run in a tab, but it occasionally fails on the first run. Trying to use the Up Arrow to re-run that task does not work. Anything that I type after that, does work.

[Maximus5]

OK. If I have a clean PC with only ConEmu installed, what shall I do to reproduce the issue?

[aarontgrogg]

From what I read from others:

• type a command
• Enter
• press up arrow Expected behavior:
• console should display previous commands entered as you press the up arrow Actual behavior:
• nothing

For me, however:

• set-up a tab to automatically run a shell script when it opens
• open that tab
• after command finishes, press up arrow Expected behavior:
• console should display previous commands entered as you press the up arrow Actual behavior:
• nothing

On Fri, Jul 31, 2015 at 11:27 AM Maximus5 notifications@github.com wrote:

OK. If I have a clean PC with only ConEmu installed, what shall I do to reproduce the issue?

— Reply to this email directly or view it on GitHub https://github.com/docker/docker/issues/13817#issuecomment-126621042.

[Maximus5]

I mean I have no docker installed or configured any of containers. Newby step-by-step guide...

[aarontgrogg]

same situation for me, actually:

• straight ConEmu
• using Startup > Task file has several lines, one calls .bat files that does: mvn lesscss:compile -Dlesscss.watch=true
• sometimes fails if /target directory isn't created yet by other tab
• pressing ^ arrow key does nothing

again, this is my situation, i think the others above are slightly different.

On Fri, Jul 31, 2015 at 11:50 AM Maximus5 notifications@github.com wrote:

I mean I have no docker installed or configured any of containers. Newby step-by-step guide...

— Reply to this email directly or view it on GitHub https://github.com/docker/docker/issues/13817#issuecomment-126629319.

[Maximus5]

Running something (mvn) from batch will not place this command in the history buffer. The reason is that the command was obtained not from user's interaction. Therefore Up arrow will do nothing.

Workarounds

1. Just restart the console, default shortcut is Win~
2. Or more tricky way, use GuiMacro print to send the command into the command input (and the history buffer)

[aarontgrogg]

gotcha, thanks!

On Fri, Jul 31, 2015 at 12:17 PM Maximus5 notifications@github.com wrote:

Running something (mvn) from batch will not place this command in the history buffer. The reason is that the command was obtained not from user's interaction. Therefore Up arrow will do nothing.

Workarounds

1. Just restart the console, default shortcut is Win~
2. Or more tricky way, use GuiMacro print to send the command into the command input (and the history buffer)

— Reply to this email directly or view it on GitHub https://github.com/docker/docker/issues/13817#issuecomment-126634830.

[crmckenzie]

@Maximus5 Re: Step-by-Step guide

There are step-by-step instructions (here)[http://docs.docker.com/windows/step_one/]

Once you have docker run hello-world working you can get to a docker container's command line by running docker run -it ubuntu bash

[phazei]

@Maximus5

Steps from clean pc with ComEmu.

1. Set "Force ConEmu as default terminal for console applications" with all checks right below it.
2. Install https://www.virtualbox.org/wiki/Downloads it's needed with boot2docker
3. Install https://github.com/boot2docker/windows-installer/releases/tag/v1.7.1 This will add the boot2docker command and docker command
   1. run boot2docker init to set up the initial virtual box it uses to run docker.
   2. run boot2docker shellinit that will give you the environmental variables to be able to run "docker" from either cmd or bash.
4. Install https://github.com/git-for-windows/git/releases/tag/v2.4.6.windows.1 This will add the bash, make sure to include the command line items to be added to path.
5. Now everything should be in a position where this issue can be reproduced.
   1. Start either cmd or bash. It should open in ConEmu
   2. run docker run --rm -it --entrypoint=//bin/bash ubuntu:14.04
      • (the first time this will need to download a large image every other time it will be fast)
      • the double // in front of bin is intentional and overcomes a msys2 feature that converts paths to windows paths :/
   3. Type anything, tap left to move cursor back, shouldn't work. Same for home/end.
   4. run exit
   5. run boot2docker ssh
   6. now run docker run --rm -it --entrypoint=//bin/bash ubuntu:14.04 again
   7. Type anything, try arrows/home/end, they work. They also work if ConEmu isn't used.

[tmds]

@Maximus5, have you been able to reproduce the issue with the instructions of @phazei?

[Maximus5]

Lack of time, haven't tried yet. Docker can't be installed in Virtual box, and I haven't got to any test PC yet. But I keep in mind this.

[gitfool]

Same problem here; running docker run -it ubuntu bash in powershell directly works as expected, but if running it via ConEmu then arrows don't work and I get other output (like @phazei):

←]0;root@3423f69d3960: /root@3423f69d3960:/# ls
←[0m←[01;34mbin←[0m  ←[01;34mboot←[0m  ←[01;34mdev←[0m  ←[01;34metc←[0m  ←[01;34mhome←[0m  ←[01;34mlib←[0m  ←[01;34mlib64←[0m  ←[01;34mmedia←[0m  ←[01;34mmnt←[0m  ←[01;34mopt←[0m  ←[01;34mproc←[0m  ←[01;34mroot←[0m  ←[01;34mrun←[0m  ←[01;34msbin←[0m  ←[01;34msrv←[0m  ←[01;34msys←[0m  ←[30;42mtmp←[0m
  ←[01;34musr←[0m  ←[01;34mvar←[0m

[lucasvc]

It also fails using docker exec -ti {container} bash.

[Maximus5]

OK, I've checked it. When you run from ConEmu

docker run --rm -it --entrypoint=//bin/bash ubuntu:14.04

The docker calls SetConsoleMode to reset flags ENABLE_PROCESSED_INPUT|ENABLE_LINE_INPUT|ENABLE_ECHO_INPUT which are critical to enable history in ReadConsole. Of course, history (up key) is not working anymore because application says "Hey, I'll process all keypressed internally, buzz off". I have no idea why it is working in cmd.exe started outside of ConEmu, it must not in theory. To process history in this mode, docker must not reset these flags. ConEmu can't do it instead of it because, when I tried it, output lines are duplicated...

root@e7570f5ed288:/#                                                                                   
root@e7570f5ed288:/# ls                                                                                
ls                                                                                                     
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var 
root@e7570f5ed288:/#                                                                                   
root@e7570f5ed288:/#                                                                                   

When you run boot2docker ssh, neither ConEmu nor started ssh client do not do history processing at all, instead all keypressed (up key too) are transferred to the ssh server, and history is processed on the server side.

[Maximus5]

@gitfool You've disabled ConEmuHk so you have unprocessed ANSI sequences in console.

[tmds]

@Maximus5 thanks for investigating!

[gitfool]

@Maximus5 Thanks. I enabled the 'Inject ConEmuHk' option and the ANSI sequences are being processed now.

Then I tried again with both Command and PowerShell consoles launched outside of ConEmu and the up key definitely provides history functionality in these cases.

[tmds]

If I understand @Maximus5's analysis, the docker executable is saying to Windows: I'll handle these special keys. I think this is okay because we expect them to be passed down to the container so that bash (or what ever is reading the terminal) can read them. So these keys arrive in the docker executable but they get lost before they reach the linux terminal.

[tmds]

The last line isn't necessarily true. ConEmu might eat them or transform them some way. I think the settings done by docker make sense and I see how they allow the history to work in cmd (bash is handling it).

[Maximus5]

As I said, docker.exe calls ReadConsole function. It read lines, so key presses like arrow keys do not reach Linux host at all. On the contrary, ssh.exe call ReadConsoleInput to read real key presses and they reach Linux host.

[tmds]

Ow I see, I misunderstood what you meant by "reset" the flags. Grepping through the docker code shows leads us to term_windows.go. In that file docker only disables ENABLE_ECHO_INPUT. There doesn't seem to be any code which would turn this on (except for restoring the previous state). ENABLE_PROCESSED_INPUT and ENABLE_LINE_INPUT do get turned on here.

[Maximus5]

Can't check it now, as I remember, docker calls GetConsoleMode several times and SetConsoleMode once during initialization. After that we do not have 0x7 flags set... they are cleared.

[koppor]

Same issue here with ConEmu 150813 and docker 1.8.3.

[tmds]

@calavera can you assign this issue to someone with the right experience? @Maximus5 is the maintainer of ConEmu and he has already investigated the issue. Now we are in need of Windows docker expert.

[thaJeztah]

@tmds thanks, I'll check with the maintainers

[thaJeztah]

@tmds @Maximus5 could you provide some additional information in what area you are seeking assistance? I briefly chatted with someone from the Microsoft team, and he wasn't sure if he could give any useful information, but they're interested in getting this issue fixed.

[Maximus5]

Docker is written in Go, isn't it? The problem is that docker.exe calls ReadConsole. This is bad approach, because even if tab-completion is working, it would be processed in the Windows side, instead of box side. It would be better to read console input, key-by-key, and transmit them to box. In WinApi there is a function ReadConsoleInput, but I'm not aware of Go.

[Maximus5]

I described the problem with ReadConsole (don't mix with ReadConsoleInput) in this comment: https://github.com/docker/docker/issues/13817#issuecomment-140941879

[lowenna]

I just spent a few minutes looking at this after talking to @thaJeztah on IRC. Using a very recent (non-public, pre-TP4) build of Windows, I added some debugging and installed the latest conEMU from http://www.fosshub.com/ConEmu.html (alpha 151025 x64).

Docker client turns off it's native ANSI emulation in the case of running under conEMU: https://github.com/docker/docker/blob/master/pkg/term/term_windows.go#L32

I put debugging https://github.com/docker/docker/blob/master/pkg/term/term_windows.go#L175 to see what the mode was. This is called at the point the client puts the terminal into raw mode. The value is 0x1E7, or binary 1 1110 0111. This equates to processed; line input; echo input on (lowest 3 bits).

Next debugging was at https://github.com/docker/docker/blob/master/pkg/term/term_windows.go#L189, and the value is 0x1E0, or binary 1 1110 0000. In other words, those three bits have been turned off (no surprise looking at the lines between).

I also added a debug statement at the entrypoint into SetConsoleMode to see if anything else was calling it, but it didn't fire apart from the client call to make the terminal raw.

I was doing this pointing docker at a Windows container (rather than Ubuntu) to see if up/down arrow did anything. What I noticed was that they didn't. I then changed docker to stop disabling ECHO_INPUT and LINE_INPUT (ie commenting out these two lines: https://github.com/docker/docker/blob/master/pkg/term/term_windows.go#L178-L179). This did have the effect of up arrow followed by return inside a (Windows) container recalling the previous command and executing it respectively. It wasn't perfect, as typing "exit" for example echo'd this out to the console.

What are the right flags that conEMU expects for the SetConsoleMode call when docker makes the console into "raw" mode? There shouldn't be an issue changing MakeRaw to set this appropriately in the conEMU case if that sounds like the issue.

[Maximus5]

May be the following would be better? Can't compile and check it now...

diff --git a/pkg/term/term_windows.go b/pkg/term/term_windows.go
index da9295e..4b529c0 100644
--- a/pkg/term/term_windows.go
+++ b/pkg/term/term_windows.go
@@ -30,8 +30,13 @@ type Winsize struct {
 func StdStreams() (stdIn io.ReadCloser, stdOut, stdErr io.Writer) {
    switch {
    case os.Getenv("ConEmuANSI") == "ON":
-       // The ConEmu shell emulates ANSI well by default.
-       return os.Stdin, os.Stdout, os.Stderr
+       // The ConEmu shell emulates ANSI on output streams well by default.
+       if IsConsole(os.Stdin.Fd()) {
+           stdIn = newAnsiReader(syscall.STD_INPUT_HANDLE)
+       } else {
+           stdIn = os.Stdin
+       }
+       return stdIn, os.Stdout, os.Stderr
    case os.Getenv("MSYSTEM") != "":
        // MSYS (mingw) does not emulate ANSI well.
        return windows.ConsoleStreams()

[lowenna]

I tried that but it just ended up hitting an invalid memory address or nil pointer dereference in the log routine.

``Administrator@WIN-9QQMSEQFG8N E:\go\src\github.com\docker\docker\docker
$ docker run -it --rm windowsservercore cmd
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x28 pc=0x48b016]

goroutine 18 [running]:
github.com/Sirupsen/logrus.(_Logger).Debugf(0x0, 0x1202bd0, 0x4e, 0xc0824d3b20, 0x3, 0x3)
e:/go/src/github.com/docker/docker/vendor/src/github.com/Sirupsen/logrus /logger.go:68 +0x26
github.com/docker/docker/pkg/term/windows.readInputEvents(0x8, 0x8000, 0x0, 0x0, 0x0, 0x0, 0x0)
e:/go/src/github.com/docker/docker/pkg/term/windows/ansi_reader.go:114 + 0x22f
github.com/docker/docker/pkg/term/windows.(_ansiReader).Read(0xc08200c720, 0xc08 24de000, 0x8000, 0x8000, 0x8000, 0x0, 0x0)
e:/go/src/github.com/docker/docker/pkg/term/windows/ansi_reader.go:73 +0 x437
io.Copy(0x4537e98, 0xc0824c3a70, 0x4537e20, 0xc08200c720, 0x0, 0x0, 0x0)
C:/Go/src/io/io.go:362 +0x1fd
net.genericReadFrom(0x4537a38, 0xc08204c5b0, 0x4537e20, 0xc08200c720, 0x0, 0x0, 0x0)
C:/Go/src/net/net.go:404 +0xbc
net.(_TCPConn).ReadFrom(0xc08204c5b0, 0x4537e20, 0xc08200c720, 0x4537e70, 0x0, 0 x0)
C:/Go/src/net/tcpsock_posix.go:72 +0xe7
io.Copy(0x4537a38, 0xc08204c5b0, 0x4537e20, 0xc08200c720, 0x0, 0x0, 0x0)
C:/Go/src/io/io.go:358 +0x144
github.com/docker/docker/api/client.func┬╖013()
e:/go/src/github.com/docker/docker/api/client/hijack.go:228 +0xda
created by github.com/docker/docker/api/client.(_DockerCli).hijackWithContentTyp e
e:/go/src/github.com/docker/docker/api/client/hijack.go:240 +0xe00
``

[lowenna]

https://github.com/docker/docker/compare/master...Microsoft:jjh/conemu?expand=1

[Maximus5]

Finally, I'd successfully built docker.exe from sources (using go build in docker folder). But now I face the API version mismatch error.

client is newer than server (client API version: 1.22, server API version: 1.20)

Any hints how to update the server?

[lowenna]

Is this a Linux server or Windows?

[lowenna]

You should be able to pull the right binary from https://master.dockerproject.org/

[Maximus5]

Linux server. Installed by DockerToolbox-1.8.3.exe.

[Maximus5]

Hmm... A how may I update the daemon on boot2docker linux server? linux server screenshot

[lowenna]

@Maximus5 I've never used boot2docker, but if it's similar to Ubuntu Linux, I do which docker to find the path of the current binary; service docker stop; cp [newbinary] /path/to/docker; service docker start. HTH :)

[tmds]

@Maximus5 alternatively you can build the docker client which supports API version 1.20. Building the 1.8.3 tag should give you a compatible client.