jahorton
commented
1 year ago My current thoughts toward this: I believe that the both the motivating problem and our existing solutions for pickers / IMX-es are similar enough to predictive text that we could implement the majority of picker support as an alternative predictive-text model type, allowing re-use of existing model-handling code.
Notable Limitations
(Initial) inability to use pickers and predictive text together
Note that initially, this would prevent use of a picker/IMX with a predictive-text model; our predictive-text worker only supports a single loaded model at present, and even if it did support a second, we'd want to support a separate thread so that ongoing predictive text operations (for a previous keystroke) don't block the picker from providing its suggestions (for a newer one) due to lack of concurrency. There's no reason we can't remedy that down the road, especially once we properly convert KeymanWeb to use of ES modules, which will greatly simplify reuse of the relevant code and more easily allow us to construct a separate worker to handle IMX operations. The design work to clearly present 'suggestions' and interaction with the user... would require significant design work and is likely a larger problem to address.
Keyman's predictive-text is currently touch-device-only; It's not yet fully cross-platform
While this point is true, note that on our roadmap, we're moving in the direction of reusing Web's OSK across all the desktop platforms. The OSK is coded in TypeScript, as is our predictive text, and the current predictive-text banner is included in that.
This is work we're already planning to do, and so the suggestion is simply to leverage these existing plans to accomplish the extra goal of IMX support as well. The predictive text banner offers a very similar UI structure to that of our existing pickers, after all.
A later point in the roadmap also mentions our desire to support predictive text on the desktop platforms; ideally, we'll be able to "two birds, one stone" that problem alongside IMX support by taking this strategy.
General Design / Implementation Strategy
Our existing predictive text model template - the Trie-based dictionary/"word list" model - assumes the following details:
- input text and output text uses the same script
- we wish to "key" each output stringwith itself, after simplifying lowercase/uppercase and "search-term keying" to provide diacritic insensitivity.
Picker "models" would reject both of those assumptions, but fortunately... these assumptions are not baked into the model design or its code interfaces. There isn't anything preventing us from mapping text in one script to output in a different script. One other notable distinction: for some cases (like with Chinese) it is possible for a one-to-many relationship to exist between source (input) text and destination (output) text. I believe that at least one of the picker keyboards above either once used or still uses a database backend to facilitate its operation.
- Using the Japanese keyboard linked above and opening Chrome's Developer mode reveals a series of API calls linked in via script:
I believe we could actually use a Trie structure similar to that for our wordlist models and adapt it for use with pickers; we'd just need to separate input and output mapping and allow leaf nodes to contain multiple possible outputs. We'd also drop the "frequency" aspect of wordlist models, providing a ordinal to keep the order of outputs deterministic.
The other interesting point of note; picker operation has its own distinct notion of 'context' that is markedly different from standard predictive-text operations; it assumes that the user will actively select entries for every "pick" made, with that acting as the "word break" our current predictive text system expects. The CJK languages don't use wordbreaking in the same way.
mcdurdin
Is your feature request related to a problem? Please describe.
In our keyboards repository, we have a few old Web keyboards that implemented custom CJK-picker functionality. This functionality is only functional on
desktopform-factor devices, and is thus not practical to use within Keyman for Android or Keyman for iPhone and iPad.See:
We've had a few recent Community forum posts highly related to this functionality, with some users and/or communities seeking to implement their own versions of this as well.
See:
Note that at their core, picker keyboards like the three existing ones listed above tend to operate based upon the following pattern, expressed roughly in Keyman language syntax:
<source text>+ [number keystroke] ><transformed text><source text>may or may not be in the same script as the<transformed text>! This is a notable distinction from our existing predictive text models, which expect same-source operation with limited transformation of the text.Describe the solution you'd like
Desired features:
Describe alternatives you've considered
No response
Keyman apps