creating ofxGestures

[tallavi]

I'm starting work on making a unified gestures handler addon that works on iOS and Android alike.

• The iOS part will be based on the work found here: https://github.com/rjraffa/ofxGestureRecognizer
• I'm going to write the android part from scratch.

It's going to have some common cpp files, with inheritance to iOS specific classes and Android specific classes that talk through JNI to OFAndroid or OFActivity.

Regarding the interface, I'm wondering if using ofEvents is the logical move here, since I'm kinda new to it.

I would like to hear some opinions regarding which option sounds like a more useful interface to you:

1) ofxGestures is a class which contains ofEvent objects. You instantiate the class, and register the events you want to methods in your app class.

2) The ofEvents objects do exist in order to pass control to the UI thread, but are private and not part of the interface. Instead ofxGestures can have an observer of type IofxGesturesObserver. The app class inherits from the observer, and implements whatever methods it want.

---

I was also thinking that I should hide from the user which view is registered to the gestures and automatically register it on the GLView. I don't think there's a reason for it to be on any other view. Sounds ok?

---

Should there be a "null implementation" for non iOS/Android systems? Or it's simply resolved by not including the addon in such systems?

---

If there are any other points you would like to raise, I'll be more than happy to read them.

Thanks,

Tal

[arturoc]

Tal, yes, i think the best would be to have a common addon that has events for the most common gestures. the particular implementations for android, iOS or others would just need to notify those events. those could be part of ofxAndroid and ofxiOS i guess and once it's working well the events could even be part of the core

[tallavi]

Isn't it wrong to create a dependency between addons?

ofxiOS and ofxAndroid seems like a more rudimentary addon than ofxGestures.

Wouldn't it be problematic to have a user who wants to include ofxAndroid to also need to include ofxGestures?

Tal

[arturoc]

there's a system of dependencies between addons, for example both ofxAndroid and i think still iOS depend on ofxAccelerometer. the project generator or the makefile system detects that dependencies and add them to the project.

[tallavi]

Wouldn't it be better to include all gesture handling related code in ofxGestures and not having the implementation be scattered also in ofxAndroid and ofxiOS?

I believe that grouping by functionality is usually more maintainable than grouping by OS.

Having said that, as this is a discussion, and I'm not that familiar with the paradigms you use in OF, I'm inclined to listen to your final word on this matter.

But if you want me to contribute into existing addons, I suggest you'll say a few extra words on what to put where.

I was thinking about adding these files:

cpp: addons/ofxGestures/[just the events, no actual implementation] addons/ofxiOS/src/gestures/[whatever files I need here] addons/ofxAndroid/src/ofxAndroidGestures.cpp [and perhaps some more related files]

Regarding java, I see there's already a file called: addons/ofxAndroid/ofAndroidLib/src/cc/openFrameworks/OFGestures

I'm going to extend it also to support the pinch event that I need. I will also take this opportunity to do a little of the refactoring we talked about. There's no reason OFGestures.java will go to the over crowded OFAndroid.java just for the JNI methods, they should be directly in OFGestures.java.

Same goes for the cpp side. I'm taking the methods out of ofAppAndroidWindow.cpp and putting them in (the newly created) ofxAndroidGestures.cpp.

sounds good?

[arturoc]

it's better to have a different addon cause usually the people who maintain android and iOS are not the same so it makes it easier for us to allow the maintainers of iOS for example to merge things without having to review to much as long as it only affects those folders.

apart from that the way you propose to organize it sounds great.

[tallavi]

I'm not sure what you are actually suggesting in your last comment. Can you elaborate?

Did you mean you want to have different addon folders called ofxiOSGestures and ofxAndroidGestures?

[tallavi]

Another contemplation - does it make sense to you that the gestures are all preregistered and are being listened to in any ofApp, and not by request?

Wouldn't it be more logical to have a method in ofxGestures that you call in order to start/stop gesture listening in the app?

[bilderbuchi]

As this is neither a feature request nor a bug report, I think it would be more appropriate to move this discussion over to the OF forum or the dev mailing list. :-)

[arturoc]

@bilderbuchi, @tallavi and i have been talking before and he proposed to add this feature and in general work on unifying iOS and android. i think it's fine to keep the discussion here since it's actually a new feature and probably not of the interest of everyone in the dev list.

[arturoc]

@tallavi for touch events we had a explicit registering method at the beginning but then decided that it was easier to just listen by default. in android the swipe gesture is already listened by default so i think it makes more sense to register to that events as soon as the ofApp is created

[bakercp]

Just to throw this out there -- I have a fairly new addon https://github.com/bakercp/ofxPointer that I use to unify all "pointer" events including mouse, touch, and pen. I began working on a gesture framework for that, but haven't made it very far yet. Currently addons like https://github.com/bakercp/ofxTouchPad (multitouch for the Mac OSX trackpad) use it, as well as my gui addons. We've talked a bit about having a unified pointer event system like this in the OF roadmap, so I'd encourage you to check out the source code. It's pretty well documented. The only gestures that it is handling currently are touch based (double touch, tap and hold, etc). So that side of things needs a lot more work / thought to unify it.

https://github.com/bakercp/ofxPointer/blob/master/libs/ofxPointer/includes/ofx/PointerEvents.h#L133-L141

Anyway, would love to see this or something like it integrated into the core at some point ...

[tallavi]

@arturoc - Gotcha. I'm on it.

@bakercp - thanks for the info, I'll have a look and comment later.

[julapy]

my 2 cents,

• i think any addon which wraps native functionality should not mix different platforms because platforms change and sometimes go in different directions.
• a unified gesture addon should be done from scratch in OF, so looking at what the mouse and touch events are doing, tracking them and matching them to a gesture.

ive made little addons like this in the past like ofxSwipe, https://github.com/julapy/ofxJulapyUtils/blob/master/src/ofxSwipe.h but a complete gesture addon would need to do this for double touch, long press, swipe, pinch and any of other crazy gestures you'd like to detect.

[julapy]

btw @bakercp, just downloaded https://github.com/bakercp/ofxPointer and will try it out on a project. a unified mouse, touch, pen event system is definitely the way forward in OF.

[bakercp]

@julapy I agree regarding writing the gesture recognizers from scratch in OF in order to keep a consistent API. The ofxPointer addon is headed in that direction ... just need more hours in the day!

[tallavi]

@julapy - I was just thinking the same thing. I started with pan and saw that I don't really need a different implementation for each OS there. Pinch and double tap are a little trickier, but I think I can recognize them properly too without the "official" OS recognizers.

Currently, I'm aiming to scrap the design I offered above and try to have a ofxGestures class that just listens to the touchDown/Moved/Up events, keeps some members, and throws back new events when it recognizes gestures. I'm starting with pan and after that pinch, since that's what I need right now. Hopefully to come back to it later and add inertia to the pinch gesture and also double tap.

I tend to agree that this approach will keep the control in OF, but I actually don't agree with your rules of the thumb. I think it's a very good idea to have a single addon that wraps the different interfaces of the different OS's and provide a single abstract layer that just works. It's just may not be needed in this case.

[arturoc]

i think using the official apis for this would be the best, we would have less code to maintain and the behaviour would be the expected in each platform. the common addon or even core would just be a new set of events that are notified from the specific implementations

[bilderbuchi]

:+1: to what arturo said, official APIs would be preferable.

[tallavi]

@arturoc, @bilderbuchi - integrating official gesture recognizers of each OS will not necessarily result in less code, and will not necessarily be the most consistent solution or the most future proof one.

The recognizers are just RECOMMENDATIONS, no one said you HAVE to go through them. Also, I have experience with integrating recognizers in iOS and Android and they have their quirks and it's not a trivial job to unite them to a single interface.

Since multi touch already works very consistently in OF (at least in iOS/Android), recognizing ourselves will give us immediate gesture support without additional work in any new OS that OF supports now or in the future (windows phone?) - provided that multi touch will work the same way there.

Anyway - I already started with the pan and it's looking very good. It's a compact and clean class. I'll post a version in a few days with pan and pinch and you can review and assess for yourself the usefulness of it.

Worst case - it will stay as an addon and not make it into the core..

[tallavi]

Well, I've pushed it! Works great, IMHO.

https://github.com/tallavi/ofxGestures

@arturoc, @bilderbuchi , @bakercp , @julapy - you are more than welcome to check it out.

Currently recognizes pan and pinch. In the future I will add double tap and inertia for pan.

Tested to work on iOS and Android, but with my personal project.

Sorry I didn't add a readme or examples and stuff..