Improved word suggestions

[GoogleCodeExporter]

If the keyboard could collect statistics about what words come after each other 
how often (http://en.wikipedia.org/wiki/Markov_chain), and use that to come up 
with suggestions, that would be great!

For example, if I tended to start texts "Hello, music lovers", the keyboard 
should keep track of that "hello" tends to be followed by "music", and "music" 
by "lovers".

This seems to be what the SwiftKey guys are doing and they are getting rave 
reviews!

Thanks otherwise for an excellent piece of work, this keyboard was a life-saver 
on my imported-from-abroad Hero when I first got it.  Good job!

Original issue reported on code.google.com by johan.walles@gmail.com on 23 Aug 2010 at 8:02

[GoogleCodeExporter]

Started:
https://code.launchpad.net/~walles/+junk/markovkbd

Change log:
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd/changes/

Get the code:
bzr branch lp:~walles/+junk/markovkbd

Submit a change locally:
bzr ci

Create a patch with your submitted changes:
bzr send -o patch.txt

Attach any patches / comments to this bug!

  Regards /J

Original comment by johan.walles@gmail.com on 27 Aug 2010 at 2:11

[GoogleCodeExporter]

Added a TODO list for things that must be done before this branch can be 
submitted for merging:
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd/annotate/head%3A/TODO-mark
ov.txt

Original comment by johan.walles@gmail.com on 28 Aug 2010 at 9:05

[GoogleCodeExporter]

Would take ownership of this if I could since I *am* working on it...

Original comment by johan.walles@gmail.com on 10 Oct 2010 at 3:33

[GoogleCodeExporter]

I have a working proof-of-concept.
1. Get the code using "bzr branch lp:~walles/+junk/markovkbd -r 31".
2. Open the project in Eclipse, build it and launch it in the emulator (or on 
your phone).
3. Type the same sentence twice ("I am a car.  I am a car.").
4. Notice how the second time you type the sentence, "am", "a" and "car" are 
all suggested without you having to type a single character.

What makes revision 31 a proof-of-concept rather than a working solution is 
that nothing is persisted when the keyboard closes or the phone shuts down, so 
you have to start over from the beginning all the time.  Working on that.

Otherwise it does work as intended, it does show what this feature is about, 
and it is ready for review / feedback on the code so far.

For a more precise list of what's missing, see the TODO list:
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd/annotate/head%3A/TODO-mark
ov.txt

Original comment by johan.walles@gmail.com on 17 Oct 2010 at 6:08

[GoogleCodeExporter]

Added another branch for unit tests:
http://bazaar.launchpad.net/~walles/+junk/markovkbd-junit/files

Tests are in this directory:
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd-junit/files/head%3A/src/co
m/android/inputmethod/norwegian/

Original comment by johan.walles@gmail.com on 23 Oct 2010 at 6:16

[GoogleCodeExporter]

Great work. Do you want your changes merged to the main branch when you are 
finished? I happy to accept code, especially since I currently don't have much 
time to work on it.

Original comment by trygv...@gmail.com on 24 Oct 2010 at 1:08

• Changed state: Started

[GoogleCodeExporter]

Absolutely, glad to hear from you Trygve!

Getting this merged has always been the whole idea.

If you want to e-mail me outside of this issue tracker I'm at 
johan.walles@gmail.com.

  Cheers /Johan

Original comment by johan.walles@gmail.com on 24 Oct 2010 at 3:06

[GoogleCodeExporter]

Revision 38 is actually testable for real (and I've just installed it on my 
phone).

Before submitting this for merging I want to at least:
1. Add a preference for disabling it.
2. Add a preference for clearing out all inter-word data.

Original comment by johan.walles@gmail.com on 31 Oct 2010 at 7:00

[GoogleCodeExporter]

To clarify that statement, it's *Bazaar* revision 38 that's a fully working 
markov-model keyboard and nothing else:
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd/files

Original comment by johan.walles@gmail.com on 31 Oct 2010 at 7:44

[GoogleCodeExporter]

Here's an alpha version of the updated keyboard.  It lacks preferences settings 
for the inter word predictions (they are always enabled) but should work and be 
stable (see below).

Please install SendLog and send log files to johan.walles@gmail.com on any 
problems.

A good test case is to type a short sentence twice and notice that you can just 
click your way through it the second time around, something like: "I am a car.  
I am a car.".

This alpha has survived about one week of real-world testing on at least two 
phones so it should be stable.  It's built from change 42 in Bazaar:
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd/files

To install it you must first uninstall Scandinavian Keyboard, because the 
unmodified Scandinavian Keyboard is signed by Trygve and this version is signed 
by me, preventing an upgrade.

Original comment by johan.walles@gmail.com on 9 Nov 2010 at 7:32

Attachments:

• testableScandinavianMarkov2.apk

[GoogleCodeExporter]

Here's an updated alpha that's much more visible, and a lot better at short 
texts.

Improvements from the last one:
* You now get suggestions for the first word in any text.
* You now get suggestions across sentence boundaries.
* Improved handling of the user pressing the physical back button on the device.

See the TODO list is for what remains (mostly preferences work):
http://bazaar.launchpad.net/~walles/%2Bjunk/markovkbd/annotate/head%3A/TODO-mark
ov.txt

Original comment by johan.walles@gmail.com on 24 Nov 2010 at 8:49

Attachments:

• testableScandinavianMarkov3.apk

[GoogleCodeExporter]

In this version the inter word relations database can grow a lot larger before 
garbage collection kicks in.

Previously the limit was at 300 relations, and that limit has now been raised 
10x to 3000.

The reason for the increase is that when my phone first GCed at the 300 mark I 
was left with only 4 relations which I feel was too little.  Having 40 sounds 
much more reasonable, so I increased the peak db size to 3000.

To do this I also had to start using the SQLite database more like a database 
and less like a plain file, so there was some more work behind this change than 
just changing a constant.

Original comment by johan.walles@gmail.com on 9 Dec 2010 at 2:25

Attachments:

• testableScandinavianMarkov4.apk

[GoogleCodeExporter]

Here's a release candidate, and assuming it survives a couple of days of 
real-world testing on a few phones I'd recommend this for merging.

Changes from the latest "testable" apk is that it has a preferences button for 
disabling the inter-word predictions, and that keyboard startup performance has 
improved.

Will add an svn diff here, until then the code can be fetched using...
bzr branch lp:~walles/+junk/markovkbd

... or browsed at http://bazaar.launchpad.net/~walles/+junk/markovkbd/files.

Original comment by johan.walles@gmail.com on 12 Dec 2010 at 9:04

Attachments:

• markovKbd-rc1.apk

[GoogleCodeExporter]

Done!

Here's the source code patch.  I'm using this live on my 2.1 HTC Hero and I'm 
very happy with it.

I'm also attaching an Eclipse project with the unit tests for the new code.  
Some of the tests test previously existing functionality as well.  Don't know 
where that should go in really.

Not allowed to change the status of this bug to something other than "Started" 
("Finished"?), would if I could.

  Cheers /Johan

Original comment by johan.walles@gmail.com on 12 Dec 2010 at 5:55

Attachments:

• markov-rc1.patch
• markov-junit-tests.tar.gz

[GoogleCodeExporter]

Here's the complete source code with my changes applied; don't know how you 
want this.

Original comment by johan.walles@gmail.com on 14 Dec 2010 at 6:58

Attachments:

• markovkbd.tar.gz

[GoogleCodeExporter]

Trygve, you mentioned earlier that you'd be happy to integrate this.

Is there anything I can do to help with that integration?

  Regards /Johan

Original comment by johan.walles@gmail.com on 7 Apr 2011 at 8:07