mz2 / MPGestures

Fork of https://github.com/fe9lix/DollarP_ObjC ported to OSX, with a supervised learning layer to improve gesture detection accuracy and backed by a crowdsourcable database of gestures
Other
21 stars 5 forks source link

$P Point-Cloud Recognizer for Objective-C

An Objective-C port of the $P gesture recognizer to be used in iOS applications.

Demo App

Github Page

What is $P?

From the $P website:

The $P Point-Cloud Recognizer is a 2-D gesture recognizer designed for rapid prototyping of gesture-based user interfaces. In machine learning terms, $P is an instance-based nearest-neighbor classifier with a Euclidean scoring function, i.e., a geometric template matcher. $P is the latest in the dollar family of recognizers that includes $1 for unistrokes and $N for multistrokes. Although about half of $P's code is from $1, unlike both $1 and $N, $P does not represent gestures as ordered series of points (i.e., strokes), but as unordered point-clouds. By representing gestures as point-clouds, $P can handle both unistrokes and multistrokes equivalently and without the combinatoric overhead of $N. When comparing two point-clouds, $P solves the classic assignment problem between two bipartite graphs using an approximation of the Hungarian algorithm. The $P recognizer is distributed under the New BSD License agreement.

Vatavu, R.-D., Anthony, L. and Wobbrock, J.O. (2012). Gestures as point clouds: A $P recognizer for user interface prototypes. Proceedings of the ACM Int'l Conference on Multimodal Interfaces (ICMI '12). Santa Monica, California (October 22-26, 2012). New York: ACM Press, pp. 273-280.

How to use

All gesture recognizer files are located in libs/DollarP. Just drag this folder into your Xcode project. See the code of the demo app on how to use the recognizer. GestureViewController.m is the implementation file where a new recognizer is added to the view:

- (void)viewDidLoad {
    [super viewDidLoad];

    dollarPGestureRecognizer = [[DollarPGestureRecognizer alloc] initWithTarget:self
                                                                       action:@selector(gestureRecognized:)];

    [gestureView addGestureRecognizer:dollarPGestureRecognizer];
}

If you want to add the default templates:

[dollarPGestureRecognizer setPointClouds:[DollarDefaultGestures defaultPointClouds]];

When a gesture is recognized, a DollarPResult object contains the name and score of the gesture:

- (void)gestureRecognized:(DollarPGestureRecognizer *)sender {
    DollarResult *result = [sender result];
    [resultLabel setText:[NSString stringWithFormat:@"Result: %@ (Score: %.2f)",
                          [result name], [result score]]];
}

Since the recognizer does not continuously calculate possible gestures, you need to call recognize at some point:

[dollarPGestureRecognizer recognize];

(Depending on your needs, you could also extend DollarPGestureRecognizer to perform the recognition in the touchesEnded delegate method.)

Adding your own templates:

See CustomizeViewController.m. You can either add preprocessed templates (DollarPointCloud) or add new templates on the fly. To add all current points as a gesture:

[gestureRecognizer addGestureWithName:name];

DollarDefaultGestures.m shows you how to add a new preprocessed template:

NSArray *points = @[
    [[DollarPoint alloc] initWithId:@1 x:0 y:100],
    [[DollarPoint alloc] initWithId:@1 x:50 y:0],
    [[DollarPoint alloc] initWithId:@2 x:50 y:0],
    [[DollarPoint alloc] initWithId:@2 x:100 y:100]
];
DollarPointCloud *pointCloud = [[DollarPointCloud alloc] initWithName:@"Roof" points:points];
[[dollarPGestureRecognizer pointClouds] addObject:pointCloud];

This would add a new gesture called "Roof" with two different strokes consisting of two points each as a new preprocessed template.

Using DollarP directly:

DollarPGestureReconizer is just a "facade" for DollarP, integrated into iOS gesture recognizers. However, you could also use DollarP directly and write your own recognizer on top of it using the methods addGesture, recognize and the property pointClouds from DollarP.h.

Notes (Demo App):