Touch scrolling should send arrow keys like how wheel events are translated

[jdmayfield]

I have a touchmove event on the terminal element that sends ansi cursor arrows down a websocket when swiped to allow easy navigation on mobile. Works great in application mode (i.e. Midnight Commander), but when on the default screen-buffer, the terminal wants to scroll. How can I programmatically disable and re-enable touch-scrolling on the terminal (without changing the buffers "scrollback")? I tried event.preventDefault(). Tried attaching the event to the terminals parentNode, its parentNode, xterm-rows, tried changing css overflow-y on a variety of elements, absolute positioning, etc. Is there an ansi code I can use for this? Some way to intercept whatever event makes the terminal "scroll"? Only solution so far is to term.scrollToBottom() every touchmove, but it jitters a lot when swiping downward. Any help would be awesome! Thank you in advance!!

Details

• Browser and browser version: Chrome 59
• OS version: Android 5
• xterm.js version: 2.8.1

Steps to reproduce

1. add touchmove event to terminal element that sends "\x1bOA" and "x1bOB" when swiped up or down.
2. test in Midnight Commander t

[jdmayfield]

Sorry, accidentally posted before I was done...

On touch-screen:

2. Test swipe in MC, cursor moves, no scrolling.
3. Test swipe in bash, history updates on cli, but terminal also scrolls.

[Tyriar]

@jdmayfield interesting use case. I think before we attempt to support something like this we should properly support ballistic touch scrolling in https://github.com/sourcelair/xterm.js/issues/594

[jdmayfield]

@Tyriar, I threw together a (sort of) module that handles touch on my system. If you guys want to use it I can post it somewhere. It might need some modification to make it more for generic use. It only sends xterm-specific escapes directly for arrow keys and scrolling, but a little elbow grease someone could update it for vt100 or to use termcap/terminfo for whatever TERM type their using. Basically, there are four swipe-regions. Bottom edge does left/right arrows. Left edge is up/down arrows. Right edge does up/down arrows when in auto-detected "mouse-mode". Main area does up/down arrows when not in mouse-mode, and wheel-scroll when in mouse-mode. Mouse mode is detected by regex matching incoming data from the websocket for mouse-mode on/off escapes (don't remember the actual names off the top of my head). Might need a mod for use with 'attach' module, as I'm not using it. I would be interested to see if anyone has a better configuration idea the "edge-scrolling" etc.

[jdmayfield]

Basically, I wanted to just make it easy to scroll. Before this project, I used Termux a lot on mobile, and I really liked the convienience of having touch-scrolling, so modelled after it. After using xterm.js, I added arrow-key and escape button overlays, also for convenience, but sometimes they get in the way, so I made the edge-swipe features.

[jdmayfield]

Whoops. Forgot to mention, I fixed the 'terminal scrolling' issue when in mouse-mode. So it'll stay put when an application expects it to just 'scroll' via xterm-escapes, not the whole terminal. Also fixed similar issue for desktop when using an real mouse-wheel.

[Tyriar]

After re-reading this issue I don't think we would want the option it's proposing as that's what application mode is for. It might make more sense in the future when the extensibility story is more fleshed out https://github.com/xtermjs/xterm.js/issues/1128

[jdmayfield]

Hmm. Not sure if I stated what I meant very well. A specific example might be clearer: I use it to interface with my Ubuntu Linode server. In a normal shell session xterm.js behaves just fine. Now let's open a large text file in mcedit, yes application mode. With a mouse your cool. But use it on your phone or tablet, the whole terminal scrolls instead-- which is not what is supposed to happen. Compatibility layer. It just makes it behave as you would expect any native application to behave on a touch-screen-- without the user having to do anything else. You wanna use the mouse you just use it, or touch you just do it. Use em both at the same time. It's seamless. Maybe someone in your group already did it for all I know, by now. I'm just saying it's worked great, making things real easy on any device without taking anything away now for about the last year.

[Tyriar]

@jdmayfield

But use it on your phone or tablet, the whole terminal scrolls instead

But if it's in application mode there's nothing to scroll? I think this is what's confusing me. Is this issue about making touch act the same as mouse in application mode then because that would be a valid issue.

[jdmayfield]

Indeed, @Tyriar, this is precisely where I am going with this. I have a whole system built around xterm.js that's it's own thing, this was just a module I added in addition to a bunch of other perks, but just this one thing would make the base a whole lot more modern and usable, and take advantage of all the touch capability inherent in pretty much everything nowadays.

[jdmayfield]

...and that without taking anything away from the mouse. Forgive me if I'm not the best with all the terminology. I scoured over every doc young or old to get info. I just mess with the escape codes and experimented with where to attach the listeners.

[jdmayfield]

I'll dig up the code for you to review this weekend. It may need some modification to work outside my system, but it was early on in development of my webapp so it won't take much.

[jdmayfield]

Here's the core of the touch scroll module from my system. It may just work out of the box, or it may need some whitling down. Either way, you should be able to see the basic logic I'm working with here. If I remember correctly, the xwheel.js module needs to be loaded in addition to the xtouch.js module for them both to work correctly. I'll try to get a more concise version for you in the next few days.

Note, in this version I am using both "scroll" escapes and "arrow" escapes. Not sure, but the listener to switch them may be in the main part of my terminal app. Most of this was actually written using my phone to connect to my cloud server over xterm.js+websocket+node.js custom https/wss server. I believe it checks the truishness of a variable to know whether to switch between application-mode type behaviour or just regular behaviour. I'll see if I can dig that up later when time permits, but basically that part just intercepted the incoming ws data looking for the application-mode signalling codes before they hit the terminal.

xtouch.js.zip xwheel.js.zip

[jdmayfield]

@Tyriar , if you would like to see how this works live, let me know and I can set you up with a login on my server.

Couple things to note:

The base workings are much simpler than I have here. I added functionality to support different "regions" on the touchscreen so you can have (but are not limited to) touch-only access to arrow keys along the edges.

The right edge uses a variable set automatically by a listener on the websocket's 'data' event that does a match() for escape codes telling it whether it's application-mode or not (I think it says mouse mode in the code though-- I experimented a lot with what seemed to work best on that one). It's not needed for the basic implementation though.

If you want just a basic version, I'll work on it. Really you don't need the xterm 'scroll' codes or to listen for the application-mode on/off codes, you can just use the arrow-key codes and forgo the logic for the edges and the scrollbar area.

I think the xwheel thing may not apply for your purposes, but I'll have to review the rest of my app to be sure.

You can replace the term.tx parts with i.e. websocket.send(xxx) . Of course, this is reliant on connection to an actual xterm-compliant server that echos your input, but I can see it's likely easy enough to adapt to a local-only xterm.js if an app is designed to only work in the browser rather than connect to a real server via websocket. Probably just send the codes straight to the terminal in that case.

I'll drop a simplified version more suitable for your purposes here soon.