Neamar / KISS

Lightning fast, open-source, < 250kb Android launcher
https://kisslauncher.com/
GNU General Public License v3.0
2.95k stars 581 forks source link

Feature request: Enter letters using gesture overlay #140

Closed ntninja closed 9 years ago

ntninja commented 9 years ago

I've been previously using "Z Launcher" and discovered your launcher on F-Droid today. While I really like how fast this launcher starts (seconds vs minutes), I also really miss the "enter letters by drawing them using gestures" feature from that other launcher. Is this something that this launcher will ever support?

Neamar commented 9 years ago

Probably not, since this would probably add too much weight to stay "light" :\

Unless there's a system feature for this? In which case it could be easily added.

Closing for now, if you find any useful resource i'll reopen!

ntninja commented 9 years ago

Depends what you consider a "system feature": There is the GestureOverlayView that can be used to record gestures draw onto the screen (giving feedback while the gesture is being draw). Using GestureLibrary the collected data then can be matched against a set of known gestures (that have to be shipped as part of the APK). The gesture library can be built using the Gesture Builder application found here: https://github.com/android/platform_development/tree/master/apps/GestureBuilder You can either compile this application yourself or trust some random person on the internet and download a pre-built version of mine here: http://storage.xmine128.tk/s/X50tpZLS7bjhnmJ :-) More information on using GestureOverlayView in general: http://stackoverflow.com/a/5579565/277882

If you accept this feature I'd definitely donate (=create) a complete Gesture library for Latin based scripts!

ntninja commented 9 years ago

Please respond whether this is anything you would ever support! Thx!

Neamar commented 9 years ago

Hi,

Sorry for the delay, haven't had much time to review all this. Following days are busy, I'll answer next week!

ntninja commented 9 years ago

No problem! Thanks for responding! I was just confused since you added the "wontfix" tag just before I had posted my response. ;-)

Some other thing I thought about: Resource titles and search terms should be de-accented to make it easier to search for applications and contacts with non-English names. Examples for some transformations that would be very useful:

Why would this useful even without gesture search?

  1. Most soft-keyboards for latin-based scripts only show the primary ASCII characters be default. Entering any of the "more fancy" characters from above usually requires long-pressing on an (un-accented) character and selecting its accented version from a sub-menu. Using the standard keyboard it takes about 2.5 to 3.5 seconds (plus maybe a bit more for interrupting the "writing flow") to enter an accented character (versus the 0.5 to 1.0 seconds for entering an ASCII character).
  2. Basically its the behavior that I (as a non-English native speaker) would expect: If I enter a the word caffe I expect to find an app that contains the word caffè in its name without bothering to add the accent. (My opinion)

Normalization is also very useful for gesture search, as keeping the individual characters apart is hard enough as it is. Was the accent pointing left or right? Or was it a badly draw circumflex or caron maybe? Maybe even a double acute? Not having to care about the accent really simplifies detection.

Neamar commented 9 years ago

(quick sidenote: normalization should already occur on KISS.

Samples: 2015-08-15 19 06 16 2015-08-15 19 08 26 )

Pluggi commented 9 years ago

If you're having troubles with normalization would you please create another issue, just to keep things nice, clear and separated ? :)

(and yeah, I have never had any problems with normalization)

ntninja commented 9 years ago

Didn't see you comment @Pluggi before! Ticket for better string normalization is now at #171.

ntninja commented 9 years ago

If I provide a working PR for gesture overlay input (after #178 has finally been merged), how big is the chance that it will be merged?

Neamar commented 9 years ago

@alexander255 first, sorry for the delay in merging your first PR, i have been on vacation lately and need to catch back on a lot of things -- sadly, KISS is a side project ;) But i'll get to your PR!

Regarding your question, i'm not a huge fan of gesture overlay, but if you have a small PR (meaning the app stays below 140k) that's definitely something i'll look at!

(althoug i may ask for a setting to disable the feature, since it may interfere with some scrolling pattern?)

ntninja commented 9 years ago

Well, it's mostly working now (but still requires some library-related tweaking). Thing is however: The gesture library (the file containing samples) will definitely be bigger than 20kB (no reasonable way around that). How about only adding the gesture detection code to KISS and making the library some separate app/download? (Also I never planned on making this feature on-by-default.)

madb1lly commented 4 years ago

Hi @alexander255, Great suggestion, I made the same myself before I found yours! Did you ever progress this? Sounds like you basically got it working, so can it be PRed? Cheers 🙂

ntninja commented 4 years ago

@madb1lly: I did “get it working”, but the whole approach using GestureLibrary turned out too simplistic and the accuracy in daily use wasn't all that great. The reason is that the algorithms used by the library are simply not up to this task (I think it uses least squares?) and quickly yield incorrect results if the input does not match any of the provided patterns very closely.

Adding it to KISS would not have been a great service to this software. I think you should really start from scratch if you want to implement this and look at some more promising ML techniques. Maybe multiclass non-linear SVM would work better? (Just an educated guess.)

madb1lly commented 4 years ago

Hi @alexander255, Thanks for the explanation, useful to know. A ML implementation doesn't sound lightweight though, so not sure it would fit in KISS anyway, sadly. Cheers 🙂

ntninja commented 4 years ago

Actually SVM is pretty lightweight (100s of lines of code) + one vs rest training data for each character (should be in the order of tens of kilobytes if properly encoded). This isn't neural networks I'm talking about here. 😀 I see how this would be some amount of work however…