[PATCH] Compose key support

[GoogleCodeExporter]

This adds support for a compose key via a long press of the dpad centre button 
(✓). The compose strings are adapted from X.org, with dead-key and non-Latin 
combinations removed (for testing reasons, as much as anything); but still 
there are ‘quite a few’ combinations.

It may make sense to put the compose sequences in a separate file; this is 
essentially a proof-of-concept patch.

Original issue reported on code.google.com by justanot...@googlemail.com on 18 Jun 2011 at 12:13

Attachments:

• compose.patch

[GoogleCodeExporter]

Hm, I have mixed feelings about this change. While the compose functionality 
makes sense on a hardware keyboard, I'm not convinced it's a good fit for the 
soft keyboard. Were you missing specific characters, or is this mainly because 
you're very used to the compose mechanism and want it to work equivalently on 
Android?

For example, the current method to get an "à" is:
- long-press the "a" key
- select the "à" character from the popup mini keyboard.

If I'm understanding your change right, the new sequence would be:
- long-press the "✓" key
- type "`"
- type "a".

It's more complicated on keyboards that don't have "`" on the main layout, for 
example for the Czech layout:
- long-press the "✓" key
- long-press the ";" key, select "`" from the pop-up mini keyboard
- type "a".

I think it's much easier to select the characters visually from a list instead 
of memorizing the combinations needed to get them. The current layout of course 
doesn't contain all possible composable combinations, but the language-specific 
layouts should have the important ones.

Original comment by Klaus.We...@gmail.com on 18 Jun 2011 at 9:43

[GoogleCodeExporter]

I'm used to Compose, but I got used to dead accent support on Acorn hw before 
that. I use both, though I tend to use Mode_shift a lot too (highly-customised 
keymap).

This patch doesn't remove the existing long-press; use whichever you prefer :-)

But it probably needs some work and some configurability. It's clone time, I 
think...

Original comment by justanot...@googlemail.com on 18 Jun 2011 at 10:50

[GoogleCodeExporter]

http://code.google.com/r/justanotherdsalt-hackerskeyboard/source/list?name=compo
se-key

Original comment by justanot...@googlemail.com on 19 Jun 2011 at 12:36

[GoogleCodeExporter]

I'm not saying your patch is wrong, I'm just a bit skeptical if it's a net 
usability improvement. Also, I've had several hairy bugs due to unexpected side 
effects of stateful key handling, and that makes me wary of adding even more 
complications to those code paths...

I think that dead keys may be worth revisiting, it seems that the text handling 
is fairly smart at least for newer Android versions. Honeycomb does interesting 
things with Arabic text and vowel diacritic combining dead keys, though 
according to an Arabic speaker it's not entirely correct yet. If I'm 
understanding the Unicode standard right (which I wouldn't bet on), the text 
handling should take care of treating a composing accent together with a letter 
as being fully equivalent to the pre-composed character, but I'm unsure if 
that's really true in practice.

Original comment by Klaus.We...@gmail.com on 19 Jun 2011 at 7:56

[GoogleCodeExporter]

Well... the combining is with the preceding glyph, but it's usual (IME) to type 
the accent first. I can see (some of) my compose code being useful here, either 
to swap things around (but that can equally well be special-cased differently) 
or to provide the equivalent single character.

'à' (U+0065 & U+0300) and 'à' (U+00E0) – definitely different, though only 
really if you're dealing with byte counts or file names…

Original comment by justanot...@googlemail.com on 19 Jun 2011 at 12:34

[GoogleCodeExporter]

I think it would be nice to support key composition like normal keyboards, just 
because people are used to it.

No special "V" or dpad key press required, simply hit the dead key followed by 
the character.

Original comment by eborrem...@gmail.com on 9 Jul 2011 at 8:33

[GoogleCodeExporter]

I think using this code to provide dead key support would be useful. There's 
partial support in the input framework for composing Unicode accents but it 
doesn't work properly, so some help from the keyboard would be needed.

Original comment by Klaus.We...@gmail.com on 16 Jul 2011 at 7:30

• Changed state: Accepted

[GoogleCodeExporter]

X's own dead keys support is implemented via the compose support.

I've updated my compose-key branch to handle some dead accents. It currently 
only recognises U+0300 to U+036F, but does work with multiple accents; and the 
compose key works within a dead accent sequence too. The following all work for 
‘ǖ’:

 dead_macron + dead_diaresis + ‘u’
 dead_macron + ‘ü’
 dead_macron + compose + ‘"’ + ‘u’
 compose + ‘_’ + ‘"’ + ‘u’

(Note that the third one, though more key presses, is identical to the second 
one: the compose sequence becomes ‘ü’ before the dead accent code gets to 
see it.)

I've not looked at the accent composition functions in the input framework. 
That would probably simplify things a bit...

Original comment by justanot...@googlemail.com on 17 Jul 2011 at 12:12

[GoogleCodeExporter]

I'll take a closer look at this as soon as I find time - I'm thinking it would 
be a nice option to provide the composing accents in the keymap, but still not 
convinced about the usefulness of a dedicated "compose" key - using the 
DPAD_CENTER (checkmark) key for that isn't really suitable since it's supposed 
to be available for other purposes, though admittedly it's not very frequently 
used.

About the input framework - the expected behavior would be that the Android OS 
should correctly normalize Unicode sequences so that a dead diaeresis + "u" 
would automatically be treated as an "ü". My experience is that this doesn't 
work correctly. Support is improving in recent Android versions but even 
Honeycomb doesn't handle accents quite right.

Short of patching the Android OS, a better alternative could be to use Java's 
java.text.Normalizer to do canonicalization within the keyboard, so that the 
Android input layer gets appropriately pre-composed characters. I haven't tried 
how well this works in practice. More here: 
http://developer.android.com/reference/java/text/Normalizer.html

However, as far as I know for some languages such as Arabic the canonical form 
uses separate non-composed characters, so this could get tricky to get right, 
though I think it would be valuable even if it only works for a subset of 
languages.

Original comment by Klaus.We...@gmail.com on 22 Jul 2011 at 6:40

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I've committed some changes which make use of this code. It's not perfect – 
some combinations don't produce correct results (combining tilde is broken, at 
least on Froyo, where there's no suitable single code point).

For valid combinations which don't work via java.lang.Normalizer, we can add 
(or restore) entries in the compose table in DeadAccentSequence.java.

Incidentally, there are valid, working combinations which are properly replaced 
but which appear as boxes. This is a font coverage issue.

Original comment by justanot...@googlemail.com on 22 Jul 2011 at 8:44

[GoogleCodeExporter]

Thank you, this is looking promising. FYI, code reviews appear switched off in 
your clone, is that intentional? Nitpick: it should work to look for 
Character.getType(...) == Character.NON_SPACING_MARK instead of checking the 
numeric ranges.

The potential lack of font coverage is expected, and is also an issue with the 
hardcoded pre-composed characters to some extent.

Original comment by Klaus.We...@gmail.com on 22 Jul 2011 at 9:46

[GoogleCodeExporter]

I've committed a change which uses NON_SPACING_MARK as described. However, I 
wonder if the other combining & enclosing marks should be handled too (though I 
note that GTK will handle them as combining for cursor movement purposes, given 
something suitable to combine them with).

(Code reviews are switched on but were restricted.)

Original comment by justanot...@googlemail.com on 22 Jul 2011 at 11:26

[GoogleCodeExporter]

I've merged this (finally!), with a few minor tweaks in revision 9b6403bb5223, 
and the latest test version includes it - try v1.23rc4 (or later) from 
http://code.google.com/p/hackerskeyboard/downloads/list . Thanks again to 
Darren for the contribution.

I've also added some true dead keys to the Danish keyboard, to help test. 
(Greek and Arabic also have dead keys, though Arabic works inconsistently on 
pre-Honeycomb systems.)

Known issues: I'm showing the dead keys using a dotted circle placeholder char, 
and that doesn't consistently compose right. I'll probably need to disable the 
placeholder on pre-Gingerbred.

Original comment by Klaus.We...@gmail.com on 3 Oct 2011 at 6:40

• Changed state: Fixed

[GoogleCodeExporter]

Forgot to add - I may end up making the compose key support conditional on an 
option if it causes issues for people, though I don't think that there's 
currently much of a reason to do long presses on DPAD_CENTER.

Original comment by Klaus.We...@gmail.com on 3 Oct 2011 at 6:41

[GoogleCodeExporter]

(Keeping the bug open for visibility, since the fix is not in the official 
release yet and several wiki pages still point to it.)

Original comment by Klaus.We...@gmail.com on 3 Oct 2011 at 7:02

• Changed state: FixInTest

[GoogleCodeExporter]

The placeholder character seems fine here (froyo), and the display issues 
aren't really a problem here. As for long presses on dpad centre: I don't know 
what might make use of short presses on it.

Original comment by justanot...@googlemail.com on 3 Oct 2011 at 10:52

[GoogleCodeExporter]

As far as the French typing habits are concerned (AZERTY layout), the typing 
order of dead key followed by the actual character is such a high-speed typing 
reflex that any other implementation would be hard to get used to!
The ê in particular is very frequent. (not to forget the other vowels, of 
course)

Original comment by bob.walr...@gmail.com on 7 Oct 2011 at 8:08

[GoogleCodeExporter]

I've updated the AZERTY keymap in revision 1266cdfe761f to add dead keys. 
Please try v1.23rc6 (or later) from 
http://code.google.com/p/hackerskeyboard/downloads/list and let me know if 
that's working as expected.

Changes to the 5-row layout:

- dead circumflex and diaeresis on the key right of "p"
  (and removed the redundant tilde there)

- dead acute and tilde as é (top row) long-press alternates

- dead grave as è (top row) long-press alternate

I've also added the dead key modifiers to the 4-row layout as long-press 
alternates on the "'" key.

Accessing the dead grave and acute accents via long-press popups is clunky, but 
that's the closest I could get to the standard layout - apparently real AZERTY 
keyboards also require the use of AltGr or similar modifiers to access those? 
Please let me know if something is misplaced.

Once the customizable key support is ready (see issue 13), it should be 
possible to provide alternative layouts, for example replacing 
less-frequently-used precomposed accented character keys with directly 
accessible dead accents.

Original comment by Klaus.We...@gmail.com on 7 Oct 2011 at 7:40

[GoogleCodeExporter]

This is included in today's v1.24 Market release (same as v1.23rc9). Let me 
know in case of remaining issues.

Thanks again to Darren for contributing the patch! (And apologies for mangling 
it with my changes ;-) )

Original comment by Klaus.We...@gmail.com on 9 Oct 2011 at 9:16

• Changed state: Fixed

[GoogleCodeExporter]

Sorry for the delay.
I haver now tested the AZERTY layout with dead key.
Excellent work, well done. For ^ and ¨ 

The long key way for the acute and grave accents is indeed clunky. Standard 
AZERTY keyboards do vary in the implementation of these diacritics. Basically 
they are not part of the french alphabet. So, they're just "nice to have".

Great work you're doing!

Original comment by bob.walr...@gmail.com on 20 Oct 2011 at 6:26

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Bulk update - changing "Fixed" to "Verified" for old bugs.

(Background: I'm changing the "Fixed" status to be considered open, the next 
steps in the lifecycle will be the closed states "FixInTest" and "Verified". 
This lets me mark issues as "Fixed" in commit messages without hiding them from 
the issue tracker.)

Original comment by Klaus.We...@gmail.com on 22 Jan 2013 at 7:33

[GoogleCodeExporter]

Bulk update - changing "Fixed" to "Verified" for old bugs.

Original comment by Klaus.We...@gmail.com on 22 Jan 2013 at 7:34

• Changed state: Verified