jawaff / KinectController

Control your computer using your body. The KinectController turns the Kinect into a controller that allows configurable actions to be emulated when configurable gestures are triggered by the user.
0 stars 0 forks source link

Allow mouse control gestures #25

Closed jawaff closed 8 years ago

jawaff commented 8 years ago

I'm not sure how this part will work yet. I will report about my ideas when I can though until I start implementing it.

jawaff commented 8 years ago

Analog reactions -- such as the mouse movement reaction -- should be able to access the Gesture object that they belong to. In addition, the Gesture object should be able to store the Rule objects, so that the analog reactions are able to lookup specific Rule objects that will instruct the reaction on what is to be done.

For example, a mouse movement reaction would be able to lookup its Gesture and then lookup the PositionX/PositionY/PositionZ and maybe Angle rules for the right hand. This would allow the reaction to figure out how fast the mouse needs to move and in what direction. Hopefully these calculations won't be too slow when they're implemented.

Anyway, I'm going to continue my research and will log any other changes to the plan for mouse movement reactions.

jawaff commented 8 years ago

A few things need to be added to the config.jj file for the mouse gesture:

  1. Esper rules for when the arm is extended.
  2. The actual rule objects that will allow a Reaction object to determine where to move the mouse.
  3. A Reaction object that will utilize the rule objects to actually move the mouse.
  4. We also need to add something to the Gesture object that will allow for the Reaction object to lookup the rule objects that are associated with the mouse movement. Note: I'm unsure how flexible we need to structure the storage of rule objects in the Gesture object, because I have no idea what other analog gestures are needed later on.
jawaff commented 8 years ago

It might make sense to start out making a child Gesture class that is specifically meant for the mouse control gesture. Then the Gesture class additions wouldn't have to be generic for any analog gesture. That may be a good or bad thing though. I need to figure out what other kinds of analog Gestures could exist in the future.

jawaff commented 8 years ago

Straight from the todos of config/config.jj:

//TODO Important: Make this gesture able to be triggered repeatedly. The Esper query would need altered.
//TODO Create a specific gesture class that is designed to handle the mouse control. It needs to keep around the
//angle rule objects and give them to the reactions as they are updated.

//TODO Look into pushing more of this configuration stuff back into some other Java files. The .jj file should be for parsing and that is it.

//TODO Create some sort of reaction that will grab the above angle rules and move the mouse.
jawaff commented 8 years ago

I've started adding a MouseReaction to the emulation package of the project and I'm unsure how to continue with this. Should I have Esper continuously triggering reaction while the user's arm is extended. Or should I make Esper in charge of turning on and turning off a 'switch' for the mouse reaction.

Required Criteria:

  1. Mouse movement should be smooth. So updated at a constant rate or the movement altered based off of the time delta (probably going to have to use a time delta.)
  2. Whatever methods we use should be able to be adapted to other gestures, to a degree at least.
jawaff commented 8 years ago

I ended up implementing PersistentReactions into the emulation package and made them able to be responsible for stopping themselves so that Esper wouldn't have to.

jawaff commented 8 years ago

So besides for a few tweaks to the smoothness and analog nature of the mouse control gesture, it is done.

jawaff commented 8 years ago

I had to fix quite a bit of things with the original setup. It was correct for the most part, but I had some trigonometry errors in the latter half of the MouseReaction class' trigger() method. Anyway, the mouse movement gesture works like a charm!