martanne / vis

A vi-like editor based on Plan 9's structural regular expressions
Other
4.27k stars 263 forks source link

Mouse support #666

Open silversquirl opened 6 years ago

silversquirl commented 6 years ago

Would be very nice if vis had mouse integration so you could scroll, click to move the cursor, etc. Currently (for me at least) scrolling moves the cursor up and down the page, which sort of works, but isn't ideal. Clicking doesn't work at all.

I recently switched from vim, and this is one of the major things I'm missing. I'm currently trying to implement it as a plugin, but I'm not sure how well that'll work.

silversquirl commented 6 years ago

I have a POC for mouse support via Lua. It's only working without curses, currently; I believe getting curses to work would require more substantial patching of vis itself.

Here's a Lua extension that will output mouse events via vis:info:

require "vis"

vis.events.subscribe(vis.events.QUIT, function ()
  io.write("\x1b[?1002l")
  io.flush()
end)
vis.events.subscribe(vis.events.INIT, function ()
  io.write("\x1b[?1002h")
  io.flush()
end)

vis:map(vis.modes.NORMAL, "<Mouse", function (keys)
  rest = keys:match(".*>")
  if rest == nil then
    return -1
  end
  vis:info("<Mouse"..rest)
  return #rest
end, "Mouse detection")

If vis is patched like this, it will also output the coordinates of the event:

diff --git a/vis.c b/vis.c
index 431c779..845b280 100644
--- a/vis.c
+++ b/vis.c
@@ -1280,7 +1280,7 @@ static const char *getkey(Vis *vis) {
    }

    TermKey *termkey = vis->ui->termkey_get(vis->ui);
-   termkey_strfkey(termkey, vis->key, sizeof(vis->key), &key, TERMKEY_FORMAT_VIM);
+   termkey_strfkey(termkey, vis->key, sizeof(vis->key), &key, TERMKEY_FORMAT_VIM | TERMKEY_FORMAT_MOUSE_POS);
    return vis->key;
 }
silversquirl commented 6 years ago

Hmmm... Turns out that actually works fine with curses.

Also, to make curses do the initialisation instead of manually outputting escape codes, it's a one-line patch:

diff --git a/ui-terminal-curses.c b/ui-terminal-curses.c
index 744cbb4..4d6ec51 100644
--- a/ui-terminal-curses.c
+++ b/ui-terminal-curses.c
@@ -271,6 +271,7 @@ static bool ui_curses_init(UiTerm *tui, char *term) {
    keypad(stdscr, TRUE);
    meta(stdscr, TRUE);
    curs_set(0);
+   mousemask(ALL_MOUSE_EVENTS | REPORT_MOUSE_POSITION, NULL);
    return true;
 }
silversquirl commented 6 years ago

I'll leave my script and ui-terminal-curses.c as they are for now. I think the next step is figuring out how to translate terminal coordinates into vis window coordinates.

silversquirl commented 6 years ago

When this snippet is placed in the mouse mapping, it will enable scrolling:

  if rest:match("^Press%(4%)") then
    vis:feedkeys("<C-y>")
  elseif rest:match("^Press%(5%)") then
    vis:feedkeys("<C-e>")
  end
martanne commented 6 years ago

As we have briefly discussed on IRC you were on the right track. Translating window coordinates to buffer positions can be done with the information available in view.c. However, it is complicated by the fact that window decorations (e.g. status and side bar) are handled externally. Also you would first have to figure out which window is affected i.e. translate screen to window coordinates.

Furthermore libtermkey doesn't properly parse mouse events from symbolic keys meaning the input handling wouldn't work as expected.

In general I agree that the mouse can be a quite powerful input device. However, I'm not sure whether a (Unix) terminal is the most suitable environment for it. In any case this will require more thought and probably some architectural changes.

silversquirl commented 6 years ago

I believe I saw some mention in the README about a possible client-server architecture with Neovim-style standalone frontends. Perhaps it would be better to wait for that to become reality and then write a graphical frontend for vis?

silversquirl commented 6 years ago

@martanne Any idea why this isn't working in INSERT mode? (I just get the names of all the mouse events inserted into my file) It works in every other mode (REPLACE included, which is somewhat puzzling).

for name, mode in pairs(vis.modes) do
  vis:map(mode, "<Mouse", mousemap, "Mouse detection")
end
Mazino-Urek commented 6 years ago

People avoids vi and uses vim because of the mouse/ touchpad support. This is 2018 and mouse support in a unix terminal can totally be justified. This is a really awesome project. Thanks.

silversquirl commented 6 years ago

@erf, @now-im I agree with both of you. Mouse support is not necessary as a core part of vis, but it would be nice to expose appropriate APIs to Lua functions in order to support it through plugins.

Personally, I would much rather use a terminal-based text editor with mouse support than a GUI editor, but alternate frontends may well be a better option for this.

mqudsi commented 6 years ago

Mouse is good if you want to quickly jump to an on-screen location in a repetitively structured/typed document without sitting there counting symbols. 99.99% of my modal text editor work is done with the keyboard, but once in a while one must admit that the mouse is the fastest way to get from point a to point b esp. without cognitive overhead.

This is an important point, imho: you're in the middle of task n (not in between tasks). To need to "calculate" where you want to land is a distraction from the task at hand, and there is a cognitive price to be paid accordingly.

Personally, when I use vis I completely forget about the lack of mouse support except once an hour or so I find myself clicking and wondering why nothing happened. So I'm not speaking as someone that can't live without the mouse - the problem is that I'm in a terminal window in a GUI and not at a text-mode tty, the mouse is there and I will reach for it subconsciously on occasion. When I do, it would be nice for it to work :)

erf commented 6 years ago

@mqudsi i see your point and i'm not really opposed to mouse support, but have you tried enabling line numbers set numbers or set nu and go to line using G9, or enabling relative line numbers with set rnu and moving to a specific line using j or k? You could also create a shortcut for turning on and off relative line numbers. I don't think it is too much cognitie overhead once you get used to this. I also noticed the amp editor has a cool feature that turns your text into tokens which you can jump to by pressing two characters - maybe this could be adopted by a vis plugin once we are able to change text using the Lua api.

daeluk commented 4 years ago

Is there any chance of revisiting this? Especially as (to me) it seems like libtermkey has implemented mouse event parsing.

erf commented 4 years ago

It seem like libtickit also have mouse support, and is more up to date.

Replacing libtermkey has been discussed before, but maybe it would make sense to replace it only for input. Perhaps it also could replace ncurses, but not quite sure about the status of it's rendering capabilites, but you can read about it here.

Perhaps better would be to abstract both rendering and input in some sort of server architecture, as mentioned here and here.

dther commented 7 months ago

Only five days away from four years of inactivity, I am pleased as punch to announce that I've finally implemented a working mouse patch that covers every usecase mentioned in this issue!

The full code is available now, in dther/vis, under the tag mouse-patch-v0, but a working diff patch which applies cleanly to 4d97ccc is available here, and soon on the wiki.

After compilation, to activate it, simply add require("vis-mouse") to your visrc, or copy the updated lua/visrc.lua for a bit of bonus functionality in the status bar.

The default behaviour should be very intuitive, but in summary:

All of this can be changed in vis-mouse.lua, which is subject to change heavily in future versions, but gives a decent overview of how I want mouse support to work going forward!

Next steps to consider before I'd be willing to write a PR:

Please leave thoughts and suggestions for how this can be further developed! I'm happy to continue working on this feature alone, but if anyone would like to assist, I'm open to collaborate.

Here's an Asciinema clip of it in action. asciicast

falsifian commented 7 months ago

Thanks! I tried it for a few minutes; two comments:

Even with broken copy and paste, this is going to be really useful to me for quickly moving the cursor.

falsifian commented 7 months ago

Oops, just realized it's completely broken in xterm without tmux.

With tmux in xterm, at least the ghost cursor and left click and left drag work.

But in bare xterm, the ghost cursor is stuck somewhere, left clicking enters visual mode (sometimes near the mouse, sometimes not), and left dragging either moves the cursor or selects some text, but it doesn't match where I dragged.

dther commented 7 months ago

@falsifian I'd only used suckless st (and tmux) to test, so I did some quick tests with xterm, urxvt and kitty. What I found was confusing, to say the least.

To be honest, these problems look like improper implementation within xterm and urxvt, given that we seem to be getting different results, and kitty has no issues. The patch relies on libtermkey for interpreting mouse sequences, and as far as I can tell, libtermkey follows the xterm specification for SGR mouse sequences strictly. When I get the chance, I'll see if having libtermkey handle initialisation will help, since the way I initiate mouse support (outputting the raw CSI sequences that activate mouse functionality) might be giving urxvt and xterm problems.

falsifian commented 7 months ago

I just did a bit of debugging, and suspect this is a bug in libtermkey, though I don't completely understand it.

I ran the following simple program:

#include <termkey.h>
#include <stdio.h>

int main(void) {
    TermKey *tk;
    TermKeyKey key;

    TERMKEY_CHECK_VERSION;
    tk = termkey_new(0, 0);

    if (tk == NULL) {
        return -1;
    }
    printf("\x1b[?1003h\x1b[?1006h");
    fflush(stdout);
    while (1) {
        int result = termkey_waitkey(tk, &key);
        switch (result) {
        case TERMKEY_RES_KEY:
            printf("got key of type %d\n", key.type);
            break;
        case TERMKEY_RES_AGAIN:
            printf("AGAIN\n");
            break;
        default:
            printf("result: %d\n", result);
        }
    }
}

I built libtermkey with DEBUG=1, and ran the above program and moved the mouse a bit. (After I exit with ^C, I run printf '\x1b[?1003l\x10[?1006l' to turn off the mouse notifications.)

With tmux within xterm, I see:

Loading the terminfo driver...
Loaded terminfo driver
Loading the CSI driver...
Loaded CSI driver
Setting termios(3) flags
Drivers started; termkey instance 0xce78e29ac00 is ready
getkey(force=0): buffer 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_NONE
getkey(force=0): buffer 1b 5b 3c 33 35 3b 38 34 3b 31 30 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_KEY
Mouse ev=3 button=0 pos=(10,84)
 mod=+00
got key of type 3
getkey(force=0): buffer 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_NONE
getkey(force=0): buffer 1b 5b 3c 33 35 3b 38 34 3b 31 31 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_KEY
Mouse ev=3 button=0 pos=(11,84)
 mod=+00
got key of type 3
getkey(force=0): buffer 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_NONE

With bare xterm, I see:

Loading the terminfo driver...
Loaded terminfo driver
Loading the CSI driver...
Loaded CSI driver
Setting termios(3) flags
Drivers started; termkey instance 0x8053b6b5300 is ready
getkey(force=0): buffer 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_NONE
getkey(force=0): buffer 1b 5b 3c 33 35 3b 36 35 3b 32 39 4d 
Driver terminfo yields TERMKEY_RES_KEY
Mouse ev=3 button=0 pos=(27,21)
 mod=C+00
got key of type 3
getkey(force=0): buffer 36 35 3b 32 39 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_KEY
Unicode codepoint=U+0036 utf8='6' mod=+00
got key of type 0
getkey(force=0): buffer 35 3b 32 39 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_KEY
Unicode codepoint=U+0035 utf8='5' mod=+00
got key of type 0
getkey(force=0): buffer 3b 32 39 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_KEY
Unicode codepoint=U+003b utf8=';' mod=+00
got key of type 0
getkey(force=0): buffer 32 39 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_KEY
Unicode codepoint=U+0032 utf8='2' mod=+00
got key of type 0
getkey(force=0): buffer 39 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_KEY
Unicode codepoint=U+0039 utf8='9' mod=+00
got key of type 0
getkey(force=0): buffer 4d 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_KEY
Unicode codepoint=U+004d utf8='M' mod=+00
got key of type 0
getkey(force=0): buffer 
Driver terminfo yields TERMKEY_RES_NONE
Driver CSI yields TERMKEY_RES_NONE
getkey_simple(force=0) yields TERMKEY_RES_NONE

Observations:

  1. With tmux, the CSI driver rather than the terminfo driver ends up handling input. The terminfo driver returns TERMKEY_RES_NONE, and the CSI seems to be used as a fallbackk.
  2. With bare xterm, the terminfo driver does find something, and the CSI driver doesn't. But somehow bytes end up left over in the buffer, so after libtermkey finds the mouse event, in re-interprets those bytes as keys pressed.
  3. As a result, in xterm, the mouse event is seen, but then some parts of the events are repeated as keystrokes.

Maybe I will report this to libtermkey.

dther commented 7 months ago

@falsifian I'VE FIGURED OUT THE PROBLEM! In short, xterm requires that all mouse modes (SGR button tracking, movement tracking) be combined into one escape code, to avoid accidentally starting any of the other incompatible mouse tracking modes. When informed to activate SGR+movement, ncurses searches terminfo for the xterm sub-entry that activates said modes, then outputs the attribute XM. In our case, that entry is xterm+sm+1006. This functionality of terminfo is an extension by ncurses which xterm uses to be compatible with a larger set of applications. See user_caps(5) for more information. Because the mouse patch outputs two distinct CSI sequences, xterm takes the second sequence as overwriting the first, causing it to detect movement, but not SGR, and in the process emitting "standard" xterm mouse codes which the plugin doesn't properly recognise. The correct fix to this is to have SGR initialisation be handled by ncurses (which I'll probably do next time I patch this) or to always emit the combined code (which should work every time, but some terminals may not support combined codes). I'll test things out when I have time.

The quick fix is to simply set TERM before running vis, like so: TERM=xterm+sm+1006 vis. When ncurses detects this special value, it suppresses the codes emitted by my initialisation function and asserts to xterm that we are in fact using both SGR mouse and 1006 (movement tracking) capabilities. In my quick tests, xterm is now fully functional, clipboard and everything.

falsifian commented 7 months ago

Hi @dther, I tried to verify some of what you said but things aren't quite adding up. I have a different explanation for what is happening.

In short, xterm requires that all mouse modes (SGR button tracking, movement tracking) be combined into one escape code, to avoid accidentally starting any of the other incompatible mouse tracking modes.

Is that fact documented somewhere?

Because the mouse patch outputs two distinct CSI sequences, xterm takes the second sequence as overwriting the first, causing it to detect movement, but not SGR,

Are you talking about the two sequences \x1b[?1003h \x1b[?1006h?

A simple experiment seems to contradict what you're saying. In xterm, I run:

printf '\x1b[?1003h\x1b[?1006h' ; cat ; printf '\x1b[?1003l\x1b[?1006l'

And then I move the mouse, and see output in the terminal like ^[[<35;41;10M^[[<35;42;10M^[[<35;43;9M just from the mouse being moved. So, it certainly seems like both escape codes are being taken into account by xterm.

(Maybe I misunderstood what you were getting at. I didn't look into what ncurses does.)

The quick fix is to simply set TERM before running vis, like so: TERM=xterm+sm+1006 vis. When ncurses detects this special value, it suppresses the codes emitted by my initialisation function and asserts to xterm that we are in fact using both SGR mouse and 1006 (movement tracking) capabilities. In my quick tests, xterm is now fully functional, clipboard and everything.

On my system, setting TERM=aoeuaoeu (or any other bad made-up value) fixes the problem in xterm. TERM=xterm+sm+1006 causes vis to behave quite badly, for reasons probably unrelated to the mouse (it doesn't seem to be able to properly draw the screen in the first place).*

I'm pretty sure the following is what is going on.

In this part of driver-ti.c in libtermkey 0.22

#ifdef HAVE_UNIBILIUM
  ti->unibi = unibi_from_term(term);
  int saved_errno = errno;
  if(!ti->unibi && saved_errno != ENOENT) {
    free(ti);
    return NULL;
  }
  /* ti->unibi may be NULL if errno == ENOENT. That means the terminal wasn't
   * known. Lets keep going because if we get getstr hook that might invent
   * new strings for us
   */
#else

if I set TERM=aoeuaoeu, unibi_from_term returns NULL (I verified that with printf debugging). As a result, libtermkey does not use the code in driver-ti.c to interpret the mouse codes. Instead, it uses the fallback code in driver-csi.c.

The driver-ti.c code is, apparently, buggy. See my previous comment. The driver-csi.c code works. So, when driver-ti.c isn't used, the problem goes away.

I maintain that this is probably a libtermkey bug.

*EDIT to add: In my /etc/termcap, xterm+sm+1006 looks like an incomplete entry, not intended to be used on its own, but used as a part of the xterm-1006 entry. Maybe your terminfo database treats it differently. In any case, I am guessing TERM=xterm+sm+1006 isn't recognized by unibi_from_term on your system, so it has the same effect as TERM=aoeuaoeu. Please do try TERM=aoeuaoeu and let me know if it works.

dther commented 7 months ago

@falsifian After further investigation, you're right, xterm detects mouse movement and SGR sequences just fine. I was wrong about how it handles control codes. My bad- my theory was based on a misreading of user_caps(5).

I can also confirm that TERM=someterminalthatdoesnthaveanentry makes the mouse work as expected- as it turns out, xterm+sm+1006 was also not meant to be used on its own in my system, so vis was falling back to default behaviour, and began properly detecting the mouse.

All that being said, from delving even deeper into the details of terminfo, I don't believe libtermkey is responsible. Vis only uses libtermkey to translate key input which is retrieved from ncurses, and that it is ncurses that is preventing the inputs from being received by the input queue.

Is that fact documented somewhere?

Not in a way that I found easy to understand. I'm learning way more about terminfo than I'd ever wished to know in the process of debugging this. I believe the key to all of this is the section XM from user_caps(5). It describes an ncurses extension to terminfo.

XM: string, override ncurses's built-in string which enables/disables xterm(1) mouse mode. ncurses sends a character sequence to the terminal to initialize mouse mode, and when the user clicks the mouse buttons or (in certain modes) moves the mouse, handles the characters sent back by the terminal to tell it what was done with the mouse.

The mouse protocol is enabled when the mask passed in the mousemask function is nonzero. By default, ncurses handles the responses for the X11 xterm mouse protocol. It also knows about the SGR 1006 xterm mouse protocol, but must to be told to look for this specifically. It will not be able to guess which mode is used, because the responses are enough alike that only confusion would result.

The XM capability has a single parameter. If nonzero, the mouse protocol should be enabled. If zero, the mouse protocol should be disabled. ncurses inspects this capability if it is present, to see whether the 1006 protocol is used. If so, it expects the responses to use the SGR 1006 xterm mouse protocol.

The xterm mouse protocol is used by other terminal emulators. The terminal database uses building-blocks for the various xterm mouse protocols which can be used in customized terminal descriptions.

The terminal database building blocks for this mouse feature also have an experimental capability xm. The “xm” capability describes the mouse response. Currently there is no interpreter which would use this information to make the mouse support completely data-driven.

xm shows the format of the mouse responses. In this experimental capability, the parameters are <snip, brevity>

Cross-referencing with the uncompiled xterm terminfo entry available here, I think our ncurses might be exhibiting slightly different, but both still "correct", behaviour. this is my current working understanding:

In short: The root cause is terminfo, not due to bugginess, but due to cruft. All of this is weird implementation greebles caused by ncurses, libtermkey and xterm trying to remain compatible with very old mouse-driven terminal software and "new" (almost 10 years old now...) experimental mouse support modes. Because of this, my naive implementation of mouse initialisation (just manually flip the bits that control mouse events) clashes with all this compatibility negotiation.

The "correct" fix is to have either ncurses or libtermkey handle the negotiation of mouse input initialisation, which isn't particularly difficult, but I foresee needing some non-trivial C logic so as to avoid annoying the other fine folks who use vis, but don't care for mouse support. Activating mouse support tends to disable terminal emulator features that many consider essential, like direct copy and paste.

I'll take all this into account in my next mouse patch, which I'm also hoping will be clean enough to be released as a PR. (I'm too full of pride/ashamed of the mess in mouse-patch-v0 to suggest merging it into master.) Thanks for your help in testing things out! It's made me reassess a lot of accidental assumptions I'd made while bug testing this.

PS: Could you try setting TERM=xterm-vt220? If my understanding of the xterm terminfo entry is correct, xterm uses xterm-new by default, which on the one hand has as many backwards-compatible features as possible, but relies on ncurses heavily for negotiating activation. xterm-vt220, on the other hand, only allows a subset, but doesn't rely on ncurses initialisation logic, and is still recognised as a valid entry that can be used on its own.

VehementHam commented 7 months ago

I like how Mouse Support is issue #666