rlwrap rewrites the input line right-shifted

[djhaskin987]

Overview

I seem to have found a bug with rlwrap.

When I type something in, then hit enter, rlwrap re-draws the input line with what I entered n spaces to the right, where n is the width of the prompt.

As an example, I will type this:

$ rlwrap ./a.out
> quit<ENTER>

And then the screen changes to look like this:

$ rlwrap ./a.out
> ququit
$ 

Steps to reproduce

1. I changed my /etc/passwd entry so that my shell was simply /usr/bin/sh. This was to eliminate zsh as the culprit.
2. I logged into the native tty of linux by hitting CTRL+ALT+F2 and logged in, so as to rule out my terminal emulator as the culprit.

3. I compiled the following C program, so as to eliminate sbcl or roswell (common lisp shells) as the culprit:

   #include <stdio.h>
   #include <string.h>
   
   int main(int argc, char *argv[]) {
    char buffer[256];
    char endline;
    for (int i = 0; i < 256; i++) {
      buffer[i] = 0;
    }
    while (strcmp(buffer, "quit") != 0) {
      for (int i = 0; i < 256; i++) {
        buffer[i] = 0;
      }
      printf("> ");
      scanf("%s", buffer);
      printf("%s\n", buffer);
    }
    return 0;
   }

4. Run rlwrap -l sesh.txt ./a.out
5. Type this<ENTER>is<ENTER>his<ENTER>face<ENTER>quit<ENTER>

You will see the following output:

$ rlwrap ./a.out

> ththis
this
> isis
is
> hihis
his
> faface
face
> ququit
quit

The file sesh.txt, however, only has this content:

[rlwrap] Fri Oct 14 19:42:57 2022

> this^M
this^M
> is^M
is^M
> his^M
his^M
> face^M
face^M
> quit^M
quit^M

(actual CTRL+M characters converted to ^ + M for readability above.)

I have confirmed that the rewrite-and-shift bug is worse for longer prompts. The shift happens 13 chars for 13 char long prompts, 6 for 6, etc.

Any pointers for workarounds would be appreciated :)

Thanks for all you do, this tool is a knockout and integral to my deverlopment workflow :+1:

EDIT:

I have confirmed this behavior does not happen when I do not use rlwrap. Should have said that up front.

I am on Fedora 36, using rlwrap 0.45.2 .

[djhaskin987]

Can confirm, the bug is NOT present in rlwrap 0.44, ncurses 6.3, and readline 8.2:

➜  cl-i git:(main) ✗ /usr/local/bin/rlwrap ./a.out 
> quit
quit
➜  cl-i git:(main) ✗ /usr/local/bin/rlwrap --version
rlwrap 0.44

[djhaskin987]

Just confirmed that the bug first appeared in 0.45(.0):

➜  cl-i git:(main) ✗ /usr/local/bin/rlwrap ./a.out 
> ququit
quit
➜  cl-i git:(main) ✗ /usr/local/bin/rlwrap --version
rlwrap 0.45

[djhaskin987]

I skimmed through the diff between 0.44 and 0.45 . On suspicion, I set the following line in my ~/.inputrc:

set enable-bracketed-paste off 

This seemed to have fixed the problem.

➜  cl-i git:(main) ✗ /usr/bin/rlwrap wraptest
> quit
quit
➜  cl-i git:(main) ✗ /usr/bin/rlwrap --version
rlwrap 0.45.2

[djhaskin987]

I found this post interesting:

8.1 enables bracketed paste mode by default now. This means from the moment when libreadline will be finally upgraded to 8.1, all Octave installations on clean systems suddenly will be affected and have to disable the 'enable-bracketed-paste' manually.

So maybe upgrading my OS upgraded readline, which enabled bracketed paste when it wasn't enabled before, exposing this bug.

[hanslub42]

I cannot reproduce this yet (I tried different recent readline versions)

The shift happens 13 chars for 13 char long prompts, 6 for 6, etc. set enable-bracketed-paste off [...] seemed to have fixed the problem

This is very useful information, thanks!! But still, it would be nice if I could reproduce this. What terminal emulator are you using? (it micght make a difference)

[hanslub42]

Your little C program has a useful little bug that in turn exposes a severe rlwrap bug: when it receives an EOF (e.g. when I press CTRL+D) it will loop indefinitely (printing "> " over and over again)

rlwrapping it and pressing CTRL+D causes rlwrap to segfault.

See #167

[djhaskin987]

Glad my shoddy work was useful :)

I am on qterminal with GNU screen running on it, with a TERM of screen-256color. But as I said, I could reproduce the bug on tty with TERM of linux.

I suspect this actually has to do with a later version of readline, which was updated in a recent version of fedora. What version of readline are you on? I'm on 8.2.

I found a post that talked about bracketed paste being turned on by default on this later version where it was originally off by default. I thought I linked it above. Maybe I didn't; I'll check.

Edit: just read your post again, you already tried this. Huh. I'm on Fedora 36. Maybe some other dependency???

[hanslub42]

The segfault was caused by the colourless_strlen() function forgetting to reset the string length to 0 when encounterin a CR

This could conceivably also be the cause of your problem, when the rlwrapped program sets the onlcr bit on its terminal, or when the terminal is initialised that way (which would mean that rlwrap receives a CR at the end of the echo'ed input even though the client program doens't emit it, and wrongly assumes it has length 0)

So, can you pull the latest devel branch and compile it? I am curious about the results.

If the problem persists, can you configure with --enable-debug (there may be some harmless warnings, cf #162), make and then

src/rlwrap -d7 ./a.out

This will write a debug log to `/tmp/rlwrap.debug. If you can send that to me it would help a lot!!

[djhaskin987]

Will do.

[djhaskin987]

New version worked:

➜  rlwrap git:(devel) src/rlwrap -d7 wraptest
>   quit
quit

➜  rlwrap git:(devel) rlwrap wraptest
> ququit
quit

➜  rlwrap git:(devel) rlwrap --version
rlwrap 0.45.2

Debug log attached. rlwrap.debug.txt

[hanslub42]

Oh, well, that was less difficult than I feared. Thanks for the report, the confirmation (and your "buggy" test program that uncovered the bug...!)

[hanslub42]

See also 82e64eba5c4198fc81fec16f30755bff5268ad62 - to make testing for this kind of cases easier