Improve key-swipe interface

[devycarol]

This is my WIP for improving the key-swipe interface.

Closes #962.

[Helium314]

Thanks for working on this! onEndSwipe(code: Int, vertical: Boolean) is currently fine, though my idea was somehow generalizing the direction. I did not get far here on how this should work, though. Angle? Or some sort of number / enum for 8 "main" directions? Provide horizontal and vertical steps or even pixels? Currenlty my favorite is using horizontal and vertical steps. The pointer tracker would not be able to tell whether we're in horizontal or vertical swipe then (but I don't see a way to get around this when we want to allow diagonal swipes).

[devycarol]

My current Bottleneck is the lack of member logic in KeyboardActionListenerImpl—there is one sListener for all the pointer trackers. Writing this I realize I could just make it non-static, but I'm not certain since I'm still a novice at Java.

Maybe there should be a separate listener for key swipes. Maybe not.

[devycarol]

I have a branch where the layout (numpad) swipe has that step minimum and doesn't trigger until pointer release. Its logic depends on knowing that initial swipe direction. An enum of the lateral directions determined onSwipeStart strikes me as a good approach. Diagonals seem untenable atm, but I could definitely see a fifth "omnidirectional" option being used by certain actions.

[Helium314]

there is one sListener for all the pointer trackers

I'm not sure about having multiple listeners. Originally the KeyboardActionListener interface was implemented in LatinIME, and was moved to a separate file to have the related logic contained. There might be issues when using multiple KeyboardActionListeners. One (simple) issue is the meta state, which could possibly be moved to KeyboardState.

I have a branch where the layout (numpad) swipe has that step minimum and doesn't trigger until pointer release. Its logic depends on knowing that initial swipe direction. An enum of the lateral directions determined onSwipeStart strikes me as a good approach. Diagonals seem untenable atm, but I could definitely see a fifth "omnidirectional" option being used by certain actions.

So maybe we should first gather what we need/want, and then choose how to do it:

• current style swipe
  • needs intial (horizontal / vertical) direction, steps in that direction, and when the swipe ends
• unexpected keyboard style swipe: 4/8 direction swipe, triggers input without releasing, repeats key, can switch direction during swipe
  • needs more precise current direction (x/y steps from starting point ideally), and when the swipe ends
• non-key-based swipe, see #532
  • probably same as current style swipe (just use fallback if the key has no swipe action), but also diagonal swipes might be useful
• "branch with that numpad swipe"
  • needs initial direction, steps, and when the swipe ends
• (possibly) showing some sort of menu on swipe, see #137, but could also be a language selector
  • not sure where this should be done, or if / how this needs to be considered in KeyboardActionListener or PointerTracker
  • would need initial (horizontal / vertical) direction, steps and when the swipe ends, I guess, but the actual selection should then be handled using views
• other ideas?

Anything I missed?

Looks like what we need boils down essentially the same for all, except the unexpected keyboard style swipe. So in case that turns out to be complicating everything, maybe it's better to drop the idea. But my impression is we can leave the option open.

So what do we need? (currently not considering there might be multiple pointer trackers)

• PointerTracker needs to know whether a key has a swipe gesture
  • basically an extended version of isSwiper, maybe with some sort of cache if necessary for performance
• Something to inform KeyboardActionListener that a swipe is in progress, and how far the movement is
  • could be onKeySwipe(keyCode, isVertical, steps) -> like now, and should be enough for most cases
  • or maybe onKeySwipe(keyCode, stepsX, stepsY) -> more flexible, and we can convert the data into the upper variant
  • can we make onKeySwipe a void and keep track of the previous steps values, or is there a good reason to keep this in PointerTracker (i.e. does the tracker need to know whether a gesture has started, or just whether it's available)
• onSwipeStarted or similar should not be necessary if we just keep track of the swipe
• Something to inform KeyboardActionListener that a swipe has ended
  • is onEndSwipe(keyCode) enough, or do we need a direction?
  • do we actually need a keyCode?
  • could we just call onEndSwipe on every onUpEventInternal?

And last... how to deal with multiple pointertrackers? Actually it might not be that complicated, if I didn't overlook something crucial. Either we need one KeyboardActionListener per pointer tracker (I don't have a good feeling about this), or we need to forward the mPointerId in onKeySwipe and onEndSwipe. With the latter, we could have an array or map of KeySwipeStates that each simply have a few vars to track the key swipe.

[devycarol]

Using the pointer ID sounds like a solid approach.

[Helium314]

I tried to sketch some idea how it could be done with onKeySwipe(keyCode, stepsX, stepsY, pointerTrackerId), but so far I don't really like it. (also, having fully focused on another app for 3 weeks after my vacation means I probably don't remember enough to produce reasonable code)

    class KeySwipeState(var keyCode: Int = 0, var inSwipe: Boolean = false, /* other stuff we may need */)
    val keySwipeStates = Array(10) { KeySwipeState() } // maybe not fixed size array, but whatever for now

    fun keySwipeAllowed(keyCode: Int): Boolean { // use a pointerTrackerId? does it make sense?
        // basically a more generic isSwiper
    }

    // this is only called if keySwipeAllowed for keyCode is true
    // returns whether startX and startY should be updated
    fun onKeySwipe(keyCode: Int, stepsX: Int, stepsY: Int, pointerTrackerId: Int): Boolean {
        val keySwipeState = keySwipeStates[pointerTrackerId]

        // we want the action for that key code
        //  it depends on code, steps, and settings
        val sv = Settings.getInstance().current
        if (keyCode == Constants.CODE_SPACE) {
            val result = if (stepsX < stepsY)
                onVerticalSpaceSwipe(stepsY)
            else
                onHorizontalSpaceSwipe(stepsX)
            if (!keySwipeState.inSwipe && result) {
                keySwipeState.inSwipe = true
                keySwipeState.keyCode = keyCode
            }
            return result
        }
        else if ...
        return false
    }

    fun onEndSwipe(pointerTrackerId: Int) { // do we need any other information if we store keyCode in swipe action?
        val keySwipeState = keySwipeStates[pointerTrackerId]
        if (!keySwipeState.inSwipe) return
        // determine up action (e.g. for delete) and do it
    }

It should work with multiple pointer trackers, but otherwise it looks like determining the action from key code and steps could get messy when adding more actions.

[devycarol]

My memory is also a bit shaky. I still care about this project, but uni will probably be keeping me rather busy in the coming weeks. You might be able to expect more contributions from me around the US Autumn holiday :)

KeySwipeAllowed has me interested in the ability to give swipe actions to any key via JSON or something, seems pretty advanced for my code skills.. I'm also curious if the layout slide ability could be converted to a keyswipe, and if key boundaries can be used in these swipe actions more generally.

For example, "only begin the action once your touch is n density pixels outside the key bounds." Imo that's probably the most sensible approach for the spacebar layout toggles and other potential gestures.

[Helium314]

My memory is also a bit shaky. I still care about this project, but uni will probably be keeping me rather busy in the coming weeks. You might be able to expect more contributions from me around the US Autumn holiday :)

Sure, I don't to steal your time!

For example, "only begin the action once your touch is n density pixels outside the key bounds." Imo that's probably the most sensible approach for the spacebar layout toggles and other potential gestures.

Such flexibility or even customizability would be great, but currently I'm afraid even my idea is a bit much. (also I'm afraid of performance impact, as stuff might get called every single frame during gestures)