KeyMagic switch issue.

[GoogleCodeExporter]

Problem: Consider the following fragment:
  < VK_OEM_MINUS > => ('eng_key')
  * + ('eng_key') => $1
Sensibly, this would mean that pressing '-' will allow you to type ONE English 
letter as the next keystroke. However, if you type:
   u-
...then you get:
   က- 
...which translates to:
   က('eng_key')
...and then, since က matches *, you get:
   က 
...which turns the 'eng_key' off. 

This affects Zawgyi and Ayar.

Original issue reported on code.google.com by seth.h...@gmail.com on 4 Jul 2010 at 12:41

• Blocking: #89

[GoogleCodeExporter]

Note: I changed the rule to:
   $uskeys = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890!@#$%^&*()-_=+[{]}\\|;:\'",<.>/?`~'
   < VK_OEM_MINUS > => ('eng_key')
   $uskeys[*] + ('eng_key') => $1
...however, this only fixes part of the problem. Consider typing two english 
letters in a row:
   -8-8
...at some point this leads to
   8-
...which resolves the same way (into nothing). 

I have two theories about this bug:
  1) I think switches should cause the rule-matching to end.
  2) I think all rules must match their switch requirements exactly. E.g., 'u' means "'u' with NO switches active", not "'u' with ANY switches active".

Original comment by seth.h...@gmail.com on 4 Jul 2010 at 1:00

[GoogleCodeExporter]

Some updates from dr. Carbon:

   1) 'a' => 'abc' will match with switches on (it doesn't mean "no switches"); however, if another rule matches with switches, then that rule takes precedence.

   2) * matches "\x{21}-\x{7d}\x{ff}-\x{ffff}" (regex)

   3) Just ensure that for:
      < VK_CAPSLOCK > => ('zg_gk')
     ('zg_gk') + * => $1 + ('zg_gk')
     ....the second rule only matches once, due to the single-letter-ascii rule (no infinite loop). Also make sure VK_OEM_MINUS behaves as expected. 

Original comment by seth.h...@gmail.com on 31 Aug 2010 at 8:43

• Changed state: InProgress

[GoogleCodeExporter]

Implemented code for (1); however, there's a pointer error. Can't test it until 
I resolve it.

Original comment by seth.h...@gmail.com on 31 Aug 2010 at 9:40

[GoogleCodeExporter]

Pointer error fixed! Should be no more random crashes. New problem: slowdown.

Checking ALL rules in all cases is slowing us down; this is necessary for 
handling switches properly.

We should be able to partition the search space by switch combinations. This 
would speed things up, since large groups of rules are never in play depending 
on the current switches pressed. 

However, before doing this, I want to make sure our semantics are correct. 
Pressing "-", for example, still does not work right. 

Original comment by seth.h...@gmail.com on 1 Sep 2010 at 10:12

[GoogleCodeExporter]

Update from dr. Carbon:
  1) Key Magic sorts rules after loading them:
     http://code.google.com/p/keymagic/source/browse/trunk/OSX/inputProcessor.m#70

We can probably sort them right before we save them to a binary format, so that 
caching hides the slowdown from sorting rules each time.

Original comment by seth.h...@gmail.com on 1 Sep 2010 at 10:25

[GoogleCodeExporter]

Paraphrasing:

When comparing two rules:
  1) If one has a greater # of virtual keys, that one is "bigger".
  2) Else if one has a greater matched "string-length", that one is bigger.
  3) Else if one has a greater switch count, that one is bigger. 
  4) Else, they're the same size.

Some notes:
  1) seems like Ctrl/Shift/etc. count as vkeys. Also, VK_* counts. 
  2) "string-length" seems to include any character and []\ under some conditions. Not sure about $x[*]. However, since we _know_ that $x[*] will only be one letter, we should be able to build up "string-length-to-match". This will also help by letting us return early if string-length-to-match is greater than curr-input-length.
  3) Switch count is easy, and this one in particular is the source of our problem.

When matching:
  1) "Biggest" goes first. 

Perhaps the easiest thing to do (after sorting, caching, etc.) is to sort 
normally, using this as a comparator, and then iterate in reverse while 
searching, then stop at the first match. This will certainly speed things up.

Not too bad... should be able to do this fairly easily.

Original comment by seth.h...@gmail.com on 1 Sep 2010 at 10:47

[GoogleCodeExporter]

Fortunately, variables and replacements are already stored separately, so I can 
sort "replacements" without affecting the variable IDs stored in "variables".

Original comment by seth.h...@gmail.com on 1 Sep 2010 at 10:56

[GoogleCodeExporter]

Implemented replacement sorting. It appears to work.
Now fixing "*".

Original comment by seth.h...@gmail.com on 1 Sep 2010 at 1:04

[GoogleCodeExporter]

Fixed "*"

Also, delayed actually setting switches until the last rule in that round had 
been matched. This seems to fit with what people will expect.

The minus key seems to work, although "--" doesn't produce a minus (will have 
to check if that's expected).

Will close this bug after checking the "caps lock" version of the minus key 
(might have to remap that to something else to test it....), and after general 
testing of the Zawgyi keyboard. 

Original comment by seth.h...@gmail.com on 1 Sep 2010 at 1:20

[GoogleCodeExporter]

Minus twice _should_ emit a minus sign.

Also, the comment section's options should use quotes (maybe optional?)

Original comment by seth.h...@gmail.com on 9 Sep 2010 at 5:29

[GoogleCodeExporter]

All right, here's the problem:
    <VK_OEM_MINUS> => ('eng_key')
    ('eng_key') + ANY => $1

Since the setter uses a VK_*, it will always be searched BEFORE the one with 
the switch. 
We can't put the switch first in the sort order, since that will break Key 
Magic's sort.
We can't search the entire list, since that leads to slowdown (as noticed 
before)

Without drastically altering our algorithm, here's what we can do:
  1) Scan the rules list and make a lookup which maps switch combinations to lists of rules (by ID, in order).
  2) If at least one switch is on (and there's a lookup for it), then search by the lookup IDs first.
  3) While doing this, maintain a vector of booleans stating if a rule ID has been checked already.
  4) If the indexed lookup fails (or never happens) then search the rules list by ID in order, skipping any rules that have their flags set to "already searched".

This will speed up typing when switches are on. Since a rule with no switches 
can mean "any switches", then the worst case still involves searching the 
entire rule base.

There's a minor issue if switches A and B are on, and the rule requires switch 
A. Does this match BEFORE rules with no requirement? What about rules which 
require only B? For now, I'm only allowing perfect matches to take precedence. 

Original comment by seth.h...@gmail.com on 9 Sep 2010 at 5:57

[GoogleCodeExporter]

Fixed. I used an unsigned int as the map index, so memory usage and lookup 
should remain... optimal.

New problem: Typing "-u-" should generate "u" and then prepare for another 
roman letter. Instead, the last press of "-" will convert the existing "u" to 
"ka". 

I have to check why this occurs. The "single ascii" rule may apply to switches, 
too. Alternatively, we might stop matching immediately after turning on a 
switch. Or, there might be something else.

I'm leaving this open until that issue is resolved, and then I'll close it and 
open new reports if Ayar/Zawgyi generate new problems.

Original comment by seth.h...@gmail.com on 9 Sep 2010 at 7:34

[GoogleCodeExporter]

KeyMagic itself is somewhat odd:

'a' => 'bbb'
'bbb' => 'ccc'
'ccc' => 'ddd'

If I type "a", then I get "bbb". If I then type "b", I get "bccc", NOT "bddd". 
This is somewhat different from how WZ does it. I will have to check to see the 
reasoning behind this....

Note that changing the order of the rules does not change the output.

Original comment by seth.h...@gmail.com on 9 Sep 2010 at 9:12

[GoogleCodeExporter]

I think I've got it... but will wait for dr. Carbon's reply.

Here's the relevant code.
if (str_out.length() > 0 && (str_out.at(0) < 0x20 || str_out.at(0) > 0x7F)){
  // Do match again
  matchRules(0, 0, KeyStates, false);
}

//So, we must break of the FIRST replacement letter is ASCII, not if it's just 
a SINGLE letter.

//Also, we break if there is NO output, which will be true if we are only 
setting a switch.

Together, these should fix our bad behavior. But, again, I'll check first.

Original comment by seth.h...@gmail.com on 11 Sep 2010 at 12:13

[GoogleCodeExporter]

Fixed. It's SINGLE ASCII + EMPTY STRING.

Any remaining bugs will require new reports; switches are basically fixed by 
this revision.

Original comment by seth.h...@gmail.com on 11 Sep 2010 at 3:19

• Changed state: Fixed