jahorton
commented
1 year ago
- Q: how to link to processEvent in a lightweight way but without breaking predictive text?
Sounds like it's time for a whirlwind tour of common/web/keyboard-processor.
OutputTarget
common/web/keyboard-processor/src/text/outputTarget.ts
This is the common base class for all forms of context supported by the Keyman Engine for Web. It includes deadkey / marker tracking:
Even predictive-text context extends this base class: the Mock class from the same file supports fully synthetic contexts and context 'cloning'.
The per-deadkey struct:
ordinal: if two deadkeys are in the same "virtual" position, this field resolves which came first.
matched: used during keyboard-rule processing.
Note that we do NOT actually insert the deadkeys into the raw text-context! Web's pattern here is different. More on that later in this comment; we do have a deadkey-inlining method that may make a useful reference.
KeyEvent
common/web/keyboard-processor/src/text/keyEvent.ts
Any and all forms of keystroke events are preprocessed into instances of that class; these instances are then passed into the Web engine's version of the KeyboardProcessor, likely similar to what would exist for core.
Note that for fat-finger correction, we pass key events that weren't pressed into the rule processing engine as well, backed by a Mock-cloned context copy.
isSynthetic
isSynthetic - true if generated through interaction with an on-screen-keyboard or for fat-finger simulation. Only false if sending the basic key-event data through to the destination, without rule processing, leads to default handling picking up the slack.
For applications of false, consider a hardware K_ENTER keystroke - this could be interpreted as a "submit form" / "search" command depending on the context.
When true, we simulate the results of "standard" default key-event handling because nothing else can handle it. At present, we don't attempt to force / synthesize an event that could trigger the "true" default.
KeyboardProcessor
The method linked above is the primary entrypoint for rule processing of individual, preprocessed KeyEvents. I doubt you'll need to inspect its internals much. It's named after Core's equivalent class, as it pretty much fills the same role. I'm almost certain this is the top-level class that an LDML engine would need to replace.
Noting the specified return type...
RuleBehavior
common/web/keyboard-processor/src/text/ruleBehavior.ts
This fellow provides a metadata perspective on what the effects of keyboard-rule processing for the KeyEvent were. As Keyman keyboards do include a few aspects of state - "variable stores", in particular - we have to keep those from taking effect for predictive-text keystrokes. They're "committed" by the next layer up from that "primary entrypoint" function, which knows which key events are real and which ones are synthesized fat-finger simulation events.
The predictionPromise field is set by that "next layer up" as well, not by keyboard rule processing.
triggerKeyDefault relates to how we handle things like word-final sigma in Greek.
I believe that it also relates to things like simple K_ENTER presses from hardware keyboards, since K_ENTER can often serve as a command-key in certain contexts. (We don't yet handle changing the intent of the OSK K_ENTER.) When embedded in the mobile apps, the mobile apps know the true context, and not the Web engine, so we rely on the host-app to make the final call on how to interpret such a keystroke. History on this aspect can be found at #5553 and its links should it prove relevant. (If it does, this'd probably be the trickiest aspect to deal with for integration with the Web engine.)
Predictive-text integration
The next layer up from KeyboardProcessor is currently named InputProcessor.
common/web/input-processor/src/text/inputProcessor.ts
This layer is the primary predictive-text driver, responsible for fat-finger emulation and forwarding of new post-keystroke context to the predictive-text engine.
This is the method that most directly receives all KeyEvent instances before forwarding them to KeyboardProcessor:
It calls this one in order to do fat-finger simulation:
Both directly include calls to the primary KeyboardProcessor instance.
On inlining deadkeys
You may find this method to be of use in regard to deadkeys:
We use this to create the 'inlined' equivalent for keyboard rules that need it, broken down into array form. For a quick example used in unit testing...
{t: 'd', ...} says "match a deadkey" here. contextCache is what the _BuildExtendedContext function will produce for a context matching the loose rule definition.
- Visible string:
ab - The two deadkeys (id 0, id 1) exist between
aandbin that specific order.
Other notes
One aspect of predictive-text that might be relevant, but probably isn't: application of a predictive-text suggestion or reversion is capable of rewinding the context to restore the "correct" context that it matches. The Transcription class (also in common/web/keyboard-processor/src/text/outputTarget.ts) is at the core of this mechanism.
mcdurdin
ermshiperete
@mcdurdin this is somewhat of an umbrella ticket. Trying to capture the discussion in issues for planning.
Support Touch Layouts in LDML.
Waiting on Joshua's 📜 merges
WASM working well, but need more WASM API endpoints into Core
main blocker: Q: how to implement core into KMFW?
Divergence in other platforms
May be able to do some test scaffolding for touch (prior to KMFW intergration)
Q: how to link to processEvent in a lightweight way but without breaking predictive text?
Note: Core in KMFW is not in v17.
🙀 #5015