Strange line editing behavior on darwin Apple Silicon

hexagonal-sun commented 3 years ago

Hi,

Now that GHC 8.10.5 has been released with Apple Silicon support, I thought I'd give it a shot. When I use ghci I'm finding issues when trying to edit the current line with the standard Emacs shortcut keys, C-a M-f, etc. To see if the issue is with ghci or haskeline, I used the example here and I managed to replicate the issue:

Suppose I've got the line (I'll use ^ on the next line to show the position of the cursor):

% foobar
        ^

I then press C-a, expecting the cursor to go to the beginning of the line, however it only jumps back one character:

% foobar
       ^

I then type hello:

%  foobahfoobaefoobalfoobalfoobaofoobar
                                      ^

Pressing Ret to read the line yields the expected value:

% foobahfoobaefoobalfoobalfoobaofoobar
Input was: hellofoobar

I've tried to replicate the issue with Darwin X86_64 and Linux X86_64 but have only seen the issue on apple silicon.

daanx commented 3 years ago

I am also observing this when going through the history where a previous entry is sometimes partially retained -- seems that "cr" is not clearing all the way back? (this is in zsh on an M1)

hexagonal-sun commented 3 years ago

Agreed. I'm also seeing issues when scrolling back through history. If I press Up or C-p to get to tht next line of history, the line isn't replaced, instead each item of history is appended to the current line.

vvulpes0 commented 3 years ago

I use Haskeline with one of my own applications, and it is worth noting that this issue only occurs when build with the terminfo flag turned on (which is the default). The issue does not exist in "dumb" mode

daanx commented 3 years ago

@vvulpes0 : ah great -- that worked for me; happy enough with the dumb terminal for now :-)

(one problem was that it seems you can only pass flags to dependencies if using stack. I used:

flags:
  haskeline:
    terminfo: false

in my stack.yaml.

The second problem is that stack was segfaulting on the M1; turned out stack starts working again when running it inside bash (or in lldb!) -- very strange. Moreover, with the latest lts-18.0 the ghc version needed is 8.10.4 but the M1 ghc is 8.10.5 so I also passed the flags stack --skip-ghc-check --system-ghc to make it work. )

hexagonal-sun commented 3 years ago

Great news. Is there a way to enable this flag for GHCi?

daanx commented 3 years ago

Unfortunately after further testing I still get very strange behavior on going up in the history -- seems like the line width becomes fixed or something giving very strange output :-(

vvulpes0 commented 3 years ago

Great news. Is there a way to enable this flag for GHCi?

i've had success with TERM=dumb ghci, though i haven't tested it thoroughly

daanx commented 3 years ago

I disabled the "scroll alternate screen" in the Preferences/Profiles/Keyboard tab -- it seems better now but it needs more testing. I am wondering if the trouble has to do with this "alternate screen" that is enabled with interactive apps like less and vim?

vvulpes0 commented 3 years ago

you can only pass flags to dependencies if using stack

i just noticed this statement, but you can pass flags to dependencies with a standard cabal build as well: when naming your dependencies' version ranges, you can also add a constraint like haskeline -terminfo to get a version with that turned off. Specifying the constraint in a cabal.project.local avoids editing the real configuration

I am wondering if the trouble has to do with this "alternate screen" that is enabled with interactive apps like less and vim?

i don't think GHCi is using the alternate screen, but i could be wrong — all of my terminals run GNU screen with altscreen disabled

daanx commented 3 years ago

Still having trouble on macOS Big Sur (M1) with readline :-(. I hope someone can look into this.

I think the tab completion goes wrong due to the "Marks" in macOS; turning off "Edit/Mark/Automatically Mark Prompt Lines" seemed to help. (these are normally displayed as little gray brackets on a tab or enter)
You cannot pass ansi escape sequences to the prompt argument in readLine -- sometimes part of that gets deleted (I think this is a Haskeline bug)
Sometimes Haskeline gets the "maxWidth" of the window wrong -- even in the dumb terminal. It seems to happen after a tab completion (?) -- this makes the columns sometimes like 5 wide which hides almost all output. If I set COLUMNS=120 it also makes it very narrow; no idea what is going on.
The behavior seems somewhat random and I cannot always reproduce...
(All of this works fine on Linux so I am guessing some escape code is working differently in macOS M1; maybe cr ?)

I tried replxx and linenoise-ng and these seem to work ok; it would be so much nicer to use a pure Haskell solution though; hope this can be fixed :-)

judah commented 3 years ago

I've uploaded haskeline-0.8.1.3 with the fix to ioctl, which might improve issues around window size (thanks to @bgamari). We use that even with the dumb terminal which might explain the behavior you saw.

I'm not sure about the issues with cr; that could be a separate issue and I'll keep investigating it.

judah commented 3 years ago

@daanx to use escape characters in the prompt you'll want to check out this documentation: https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt

judah commented 3 years ago

For the cr issue, I suspect there's some difference in stty behavior. We can probably work around it by setting the right terminal mode: https://downloads.haskell.org/~ghc/latest/docs/html/libraries/unix-2.7.2.2/System-Posix-Terminal.html#t:TerminalMode

Could someone with that issue please do the following steps:

let me know what version of MacOS and of Terminal.App you're using
In your terminal, run stty -a and paste the results here
Start ghci, run !stty -a from within it and paste the results here

daanx commented 3 years ago

Awesome -- just compiled with the new version and everything seems to work fine now. There seems no further problems with cr either.

I guess that the ioctl was just going wrong on the M1 due to the variable arguments and that that caused the random behavior.

I also applied your hint with appending STX to escape sequences and it works as well -- thanks! (although, it would be nicer if Haskeline would recognize CSI sequences terminated with m so this is not needed).

hexagonal-sun commented 3 years ago

Great. Looks like we can close this ticket? Thanks all!

judah commented 3 years ago

Glad to hear it's resolved; thank you everyone for your help tracking this down!

minoki commented 3 years ago

I investigated the cr issue on my own and just discovered this issue. On my Mac, https://github.com/judah/terminfo/pull/41 seems to fix it.

lehmacdj commented 3 years ago

I still appear to get this issue on my Apple Silicon Mac when not using TERM=dumb. As an example here is a small repl that can reproduce the issue on haskeline-0.8.2:

Program source

```haskell #!/usr/bin/env cabal {- cabal: build-depends: , base ==4.14.3.0 -- ^ for pinning ghc-8.10.7 , haskeline ==0.8.2 -} -- | Simple example program for demo-ing problems with haskeline line editing on -- M1 mac. module Main where import System.Console.Haskeline repl :: InputT IO () repl = do mline <- getInputLine "> " case mline of Nothing -> pure () Just line -> outputStrLn line *> repl main :: IO () main = runInputT defaultSettings repl ```

I get the same line editing behavior that was originally reported both in the simple example program and GHCi:

history appends instead of clearing
actions like C-w only delete one character instead of the whole word as expected
motions like C-e (or 0 in vim mode) only move one character back

I also tried compiling with terminfo: false on a project of mine and the problem remained as well.

When using TERM=dumb all of these issues go away, but ideally this should work "out of the box" without requiring setting TERM=dumb manually.

Here is additional troubleshooting info that you asked for earlier on the thread:

stty -a

``` speed 38400 baud; 32 rows; 185 columns; lflags: icanon isig iexten echo echoe echok echoke -echonl echoctl -echoprt -altwerase -noflsh -tostop -flusho pendin -nokerninfo -extproc iflags: -istrip icrnl -inlcr -igncr ixon -ixoff ixany imaxbel iutf8 -ignbrk brkint -inpck -ignpar -parmrk oflags: opost onlcr -oxtabs -onocr -onlret cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow -dtrflow -mdmbuf cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = ; eol2 = ; erase = ^?; intr = ^C; kill = ^U; lnext = ^V; min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W; ```

:!stty -a output in GHCi

``` speed 38400 baud; 32 rows; 185 columns; lflags: -icanon isig iexten echo echoe echok echoke -echonl echoctl -echoprt -altwerase -noflsh -tostop -flusho -pendin -nokerninfo -extproc iflags: -istrip icrnl -inlcr -igncr ixon -ixoff ixany imaxbel iutf8 -ignbrk brkint -inpck -ignpar -parmrk oflags: opost onlcr -oxtabs -onocr -onlret cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow -dtrflow -mdmbuf cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = ; eol2 = ; erase = ^?; intr = ^C; kill = ^U; lnext = ^V; min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W; ```

lehmacdj commented 3 years ago

Sorry, false alarm! judah/terminfo#41 fixed this for me too

dixonary commented 2 years ago

For those of us stuck using existing binaries, I'm happy to report that unlike TERM=dumb, setting TERM=linux seems to resolve most/all of the readline problems and allows for useful properties like screen-clearing.

Davidhervil commented 1 year ago

Similar issues happening on M1 Pro with ghci. Can't use arrows to move along ghci REPL without characters getting mangled. They are correctly input since my muscle memory writes the correct code I want and executes fine but when I see the terminal it's just pure nonsense 😆 Though when doing detail editing it's just impossible follow along

haskell / haskeline

Strange line editing behavior on darwin Apple Silicon #162