selection mode highlighting is very confusing in my terminal

[njsmith]

This is with gnome-terminal on Linux, using ipython.

Suppose I have this text:

here is the mark: _ more text

I place my cursor on top of the underscore, and hit control-space, and then move my cursor around to perform various experiments:

What I see after moving my cursor to the left a bit (using [x] to indicate that x is displayed in reverse-font):

here is the mark[:][ ][_] more text

What this means: the text k: _ is selected, and will be cut if I hit control-W. Notice that k is not highlighted.

What I see immediately after hitting control-space, before moving the cursor anywhere:

here is the mark: _ more text

i.e., the cursor has disappeared, and nothing is highlighted. What this means: the text _ is selected, and will be cut if I hit control-W. (NB that in addition to being visually confusing, this is subtly incompatible with what emacs does, cf. #323 -- for emacs, immediately after hitting control-space, the mark and point are both immediately to the left of the character that's under the cursor, and so hitting control-W cuts the empty string.)

What I see after moving the cursor one space in either direction:

here is the mark: [_] more text

What this means: either the text _ (space and the following underscore) or the text _ (underscore and the following space) is selected and will be cut if hit control-W. These cases are visually indistinguishable.

What I see after placing the cursor top of the t at the end of the line:

here is the mark: [_][ ][m][o][r][e][ ][t][e][x]t

What I see after moving the cursor one more space to the right, so that it's actually at the end of the line:

here is the mark: [_][ ][m][o][r][e][ ][t][e][x][t][ ]

i.e. moving the cursor one space to the right apparently causes two extra highlighted spaces to appear. AFAICT these two cases are identical in terms of what's selected: in both cases, hitting control-W will cut the text _ more text (which is not what's visually selected in either case!)

I think maybe there are two underlying bugs here:

• prompt-toolkit should follow emacs and interpreter the mark and point as being placed at the space in between characters, to the left of where the cursor is when control-Space is pressed (for mark) and where the cursor is currently (for point)
• prompt-toolkit's attempt to highlight the selected region by reversing the text colors collides with gnome-terminal's using this same approach to show the cursor. It would be better if prompt toolkit used some more distinctive mechanism for showing the highlight, like perhaps the light grey background that it uses for completion drop-downs.

[asmeurer]

I noticed this too. It means that if you select a line with Control-space + Down arrow and then cut it, it deletes the first character of the next line. This is not how selection works in emacs, but I just checked and apparently it is how it works in vim.

[asmeurer]

The fix is really easy too:

diff --git a/prompt_toolkit/document.py b/prompt_toolkit/document.py
index 25d817d..422a491 100644
--- a/prompt_toolkit/document.py
+++ b/prompt_toolkit/document.py
@@ -857,8 +857,8 @@ class Document(object):
                     new_cursor_position = from_

                 remaining_parts.append(self.text[last_to:from_])
-                cut_parts.append(self.text[from_:to + 1])
-                last_to = to + 1
+                cut_parts.append(self.text[from_:to])
+                last_to = to

             remaining_parts.append(self.text[last_to:])

[asmeurer]

Actually the selection itself is correct for what it is doing. This can be confirmed visually if you select text before the cursor.

The problem is that selection is treating the cursor as if it were after where it is. But in emacs, the cursor is treated as if it were before where it is (think in terms of a modern GUI where the cursor is in fact between characters).

In fact, if you change the terminal cursor to a vertical bar in iTerm 2 or in Terminal.app (assumedly also in Linux terminals if they support that), the bar is placed before the cursor character. vi's behavior in visual mark mode becomes a bit odd if you do this. v left selects what (appears) to be the character after the cursor.

Anyway, vi is older than I am, so I'm not going to argue it's logic. Apparently some people like it. But,

1. The default behavior of prompt-toolkit already matches emacs in most places,
2. The very keyboard shortcut Control-space comes from emacs. But pressing control-space in prompt-toolkit selects a character (the character under the cursor).
3. In fact, in vi_mode, the v also selects a single character, which is different from vi! In vi (as in emacs), v creates an empty selection.

So I think there are two issues:

• Control-space / v should create an empty selection
• In emacs (and the default) mode, starting a selection and moving left or up should not select the character under the cursor. Or, rather, once the above point is fixed, in vi mode, doing so should select the character under the cursor (since just v would create an empty selection). Actually I think there might be some other weirdness with vi as well depending on whether you go left or right, so we should just try to replicate it.

[jonathanslenders]

Hi Nathaniel and Aaron, Thanks a lot for this extensive analysis! I did not had the time yet to focus on this issue. Right now, the focus is mainly on getting 2.0 released. But in any case, I consider this a bug, which needs to be fixed.

[asmeurer]

Since this is somewhat of a backwards incompatible change, should this be considered for 2.0?

[jonathanslenders]

I think this should fix it: https://github.com/jonathanslenders/python-prompt-toolkit/pull/619 Would you mind trying that fix?

[jonathanslenders]

This fix has been merged: https://github.com/jonathanslenders/python-prompt-toolkit/pull/619 I think it should be fine now. Please let me know if I miss something. It doesn't do the whole kill-ring thing yet, but that's a different issue.