search

directvt / vtm

Text-based desktop environment
MIT License
1.62k stars 43 forks source link

Mouse wheel is not supported in application mouse tracking modes #62

Closed ismail-yilmaz closed 3 years ago

ismail-yilmaz commented 3 years ago

When a window got focus in application mouse tracking modes, it should also allow mouse wheel events. To reproduce simply run a CLI app, such as htop, or vim and try using mouse wheel. It does nothing.

For more info on mouse wheel support, see: https://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h3-Wheel-mice

Tested terminals: xterm, gnome-terminal, kitty Tested terminal widgets: vte, TerminalCtrl (Ultimate++ terminal widget) Tested platform: Linux 5.10, Gnome 3.38.

o-sdn-o commented 3 years ago

vim(:set mouse=a) and htop request CSI ? 1000 h SET_VT200_MOUSE

image

Currently only CSI ? 1006 h SET_SGR_EXT_MODE_MOUSE protocol is supported, I started making support for 1000-1003 modes as well as 1006 😊

ismail-yilmaz commented 3 years ago

Ok, thanks. It is nice to see this tool is getting better each day.

o-sdn-o commented 3 years ago

Done. X11 mouse tracking protocol CSI ? 1000 h is now supported by vtm.

I have also updated the release for linux.

o-sdn-o commented 3 years ago

NASA's Mars 2020 Perseverance Rover Landing live broadcast begins 🤞

o-sdn-o commented 3 years ago

Do you think the modes 1001-1003 should be implemented, or 1000 will suffice? What applications use these modes and are they even needed in the current reality?

Now when an application requests any of these modes, it uses mode 1000 instead. I don't know if this behavior is enough.

ismail-yilmaz commented 3 years ago

Do you think the modes 1001-1003 should be implemented, or 1000 will suffice? What applications use these modes and are they even needed in the current reality?

Now when an application requests any of these modes, it uses mode 1000 instead. I don't know if this behavior is enough.

No, mode 1000 will not suffice as it only reports mouse button press and release. It does not report motion. (at least it shouldn't).

The problem I've encountered with your multiplexer is not that it uses mode 1006. In fact it is the recommended mode. All the mode 1006 does is translate the terminal response for the modes 1000, 1002 and 1003 to SGR format. Hence SGR mode should also report wheel ups and downs (they are buttons 4 and 5, or 64 and 65 in SGR mode). (Except for the legacy X10 mouse tracking mode `9', which is deprecated anyway)

See this recommendation (as you know, this is still a draft but these recommendations are based on real usages.) It even recommends a precedence among these modes, which works very well in real life scenarios.

o-sdn-o commented 3 years ago

Thank you, now it's all clear 😊

The X11 is a default mouse reporting protocol. 1006 only switches the default mouse reporting protocol to SGR.

~1000 mode for buttons tracking~ ~1001 mode for buttons + movements while pressed tracking~ ~1002 mode for buttons + movements tracking~

1000 mode for buttons tracking 1002 mode for buttons + movements while pressed tracking 1003 mode for buttons + movements tracking

ismail-yilmaz commented 3 years ago

Thank you, now it's all clear blush

The X11 is a default mouse reporting protocol. 1006 only switches the default mouse reporting protocol to SGR.

1000 mode for buttons tracking 1001 mode for buttons + movements while pressed tracking 1002 mode for buttons + movements tracking

Yes, this basically sums up the modes. But there is one thing that is usually overlooked: The default mouse coordinates mode on terminals limit the resolution to 256x256 cells, as it uses a byte oriented protocol. There were other coordinate formats to surpass this limits (mode '1005 - UTF8and mode '1015 - URXVT), but they are deprecated too. SGR does not have this limit, you can represent coordinates >= 256, as it is a string based protocol. So I would recommend explicitly clamping the mouse coordinates to (0-255) when you are not using the mode 1006 and disable the upper limit (or set a sane upper limit, i.e. 64K) when you are using the mode 1006. Forgetting this leads to weird effects on terminal displays >= 256 cells.

o-sdn-o commented 3 years ago

So I would recommend explicitly clamping the mouse coordinates to (0-255) when you are not using the mode 1006

That's right, coordinates are already being cut https://github.com/netxs-group/VTM/blob/e1b36187f744849dcb9e1ea0b8d67ac5d7cc97fc/src/netxs/console/ansi.hpp#L331-L335

I'm worried about the following question (I can't find the answer anywhere): Why are all developers against negative parameter values ​​in CSI? This also applies to the SGR mouse reporting protocol (eg CSI < 32; -10 ; -5 m) and CSI in general.

Could this be due to the use of some public ANSI VT sequence parsing algorithm that does not allow negative numbers?

I need to use negative numbers when working with the mouse, when the cursor goes out of the window when dragging. I don't know if I should do my own new mouse tracking mode or just allow negative numbers.

Example: https://www.youtube.com/watch?v=nIujUzl30NI

EDIT:

Standard ECMA-48: Control Functions for Coded Character Sets

5.4 Control sequences

The format of a control sequence is CSI P ... P I ... I F where

  • CSI is represented by bit combinations 01/11 (representing ESC) and 05/11 in a 7-bit code or by bit combination 09/11 in an 8-bit code, see 5.3;
  • P ... P are Parameter Bytes, which, if present, consist of bit combinations from 03/00 to 03/15 (ASCII 0123456789:;<=>?);
  • I ... I are Intermediate Bytes, which, if present, consist of bit combinations from 02/00 to 02/15 (ASCII space and !"#$%&'()*+,-./);
  • F the Final Byte.

This standard does not provide for the minus sign as Parameter Bytes.

But everyone assumes that the CSI numeric parameters are a series of semicolon-separated literal numbers, not bytes

Therefore, it is possible to violate (use negative numbers) this standard without spoiling the expected behavior. What do you think?

ismail-yilmaz commented 3 years ago

This standard does not provide for the minus sign as Parameter Bytes. But everyone assumes that the CSI numeric parameters are a series of semicolon-separated literal numbers, not bytes

I suggest you read, if not already did, the DEC STD070 document, aka "the bible".

It explicitly defines the wire formats (ESC,CSI, DCS, OSC, APC, SOS, PM), albeit inconsistent with the later documents or its revisions on some points. See especially the page 3-23 Extension layer (#124 and on in pdf). You'll hopefully find the answer for this question. It explains the specs.

Therefore, it is possible to violate (use negative numbers) this standard without spoiling the expected behavior.

Minus or hypen (0x2d) is reserved. It is in the range of intermediary bytes. (0x20-0x2f) for CSIand 'DCS' sequences. So using it it will break the parsers and violate the protocol. However, if you want to use it internally, you can simply use an PM (Private message, allocated for private use, if you are going to use it internally. And It allows to define your own protocol.

Still, if you are going to define a new command, which I highly recommend avoid using CSI' or 'DCS for this, you can represent negative and positive numbers by using another parameter, which takes 1 for positive, 0 for negative, for example.

ismail-yilmaz commented 3 years ago

OTOH, in theory you can use negative numbers in a new OSC function. It accepts 0x20-0x7f as parameter. Still, I am not sure if it would be considered good practice to do so. You can prepend or append P and N to numbers too.

E.g. let's assume that we request from the host app a sum of two signed integers, sum(10, -20) This can be represented as:

If this function is going to be used internally, as I wrote on my previous message, then you should replace OSC with PM and its specs.

P.S While the OSC initally conceived as having reserved intermediary bytes, in practice, to my knowledge, this has never been actualized.

o-sdn-o commented 3 years ago

Thanks a lot for the link to the DEC STD070 document and helpful thoughts!

With CSI sequences everything is very clear - negative numbers in CSI cannot be used in general.

BUT, from DEC STD070 document:

  • If 3/0 to 3/15 occur after the occurrence of one or more intermediate characters, the sequence is invalid. However, the end of the control sequence is still defined by the final character. Interpretation will continue until a valid final character is received.

This means there will be no side effects when the SGR mouse report with a minus sign is received, no extraneous characters in the input, just the whole sequence will be discarded, which is great.

Now I really want to add a minus sign to the SGR mouse reporting protocol.

Now I really don't want to come up with a new sequence for this, and I'm tempted to take the risk and break the existing order.

Apps that can't interpret the minus sign, they also won't know the new mouse protocol if I have to reinvent it, so this functionality won't be available to them anyway.

o-sdn-o commented 3 years ago

It's not completely clear to me what the consequences/side effects may be, but so far everything looks optimistic.

Probably the biggest problem is that applications can assume that the terminal always emits valid sequences, and with this in mind, make some optimizations.

o-sdn-o commented 3 years ago

A sub-mode is required to enable sending negative coordinates in SGR reports.

For example, the MidnightCommander parser does not discard the entire sequence when it receives a minus sign

https://github.com/MidnightCommander/mc/blob/4ae6cdb774105c6ee7754c19b025f1723ec45ddc/lib/tty/key.c#L775-L798

and extraneous characters appear in the input.

EDIT: It will be something like

I will now test all popular terminals for side effects when using this sub-mode.

o-sdn-o commented 3 years ago

The idea with sub-modes did not work. Terminals do not consider 1006:n value to be equivalent to plain 1006. So I have to do my private mode CSI ? 1003 ; 1006 ; 99999... h.

It is going to be a 10060 mode:

o-sdn-o commented 3 years ago

Now negative parameters will only be sent to applications that explicitly ask for it using CSI ? 10060 h.

I've updated the releases.

ismail-yilmaz commented 3 years ago

@o-sdn-o ,

As you can see, not all terminals and apps implement the sequences and follow the rules strictly. :) In an ideal world, unrecognized or irregular sequences would be ignored by all terminal emulators or apps. In practice, however, they simply breaks some of them; even valid but unrecognized sequences break them (e.g initial implementation of DEC's DCS sequences by some popular apps caused a lot of trouble some time ago.). And it is hard to change established behavior. Still, IME, the sane route to take would be to concieve a private sequence and corresponding mode, if you really need one. The other factor is both VT and CLI app developers would be reluctant to introduce a new sequence and mode., for good reasons.

I will have the time to check the release tomorrow, and provide more feedback.

Cheers

o-sdn-o commented 3 years ago

Drag-scrolling is one of the key reasons for using the DECSET 10060 Extended SGR mode.

Using the DECSET 10060 Extended SGR mode vtm drag-scrolling

Without using the DECSET 10060 Extended SGR mode vtm wo_drag-scrolling

o-sdn-o commented 3 years ago

Everything here seems to be done and works fine, but if something is wrong, I'll reopen this issue.