We don't seem to have bigram data when the word includes an apostophe

KomodoOpenLab / TypeOver

TypeOver facilitates typing on your iOS device when using external switch interfaces compatible with VoiceOver.

1 stars 2 forks source link

We don't seem to have bigram data when the word includes an apostophe #206

Open tnantais opened 11 years ago

tnantais commented 11 years ago

I'm wondering if there's an issue in the XMLtoSQLite import code where words with apostrophes are rejected.

OwenMcGirr commented 11 years ago

There should not be but I'll explore.

On Tue, Sep 10, 2013 at 2:20 PM, tnantais notifications@github.com wrote:

I'm wondering if there's an issue in the XMLtoSQLite import code where words with apostrophes are rejected.

— Reply to this email directly or view it on GitHubhttps://github.com/KomodoOpenLab/TypeOver/issues/206 .

Thanks, Owen McGirr

tnantais commented 11 years ago

I think it's possible that the apostrophe problem in XMLtoSQLite is causing multiple identical bigrams to appear in the database, e.g., I suspect "I'd,think" is going in as "I,think". This is a problem for the prediction algorithm, which expects each bigram to appear once.