Demo: https://www.youtube.com/watch?v=oH0ZkfFoeYU
This program uses Python 2 and OpenCV 2 to attempt to detect the user's hand gestures. Gestures can be mapped to different actions and new gestures can be trained.
Currently works best in low lighting with the hand as the brightest portion of the image.
Features:
Yet to be implemented (in no particular order):
Linux: Required modules are probably bundled with your favorite distribution of Linux (i.e. Ubuntu, Debian, Linux Mint, etc.) However, in the event that it needs to be installed, the following instructions can be followed:
sudo apt-get install python
sudo apt-get install python-numpy
See testImplementation.py
for example.
current_src
directory into your project's working directory (to allow easy importing of modules)GestureProcessor
in the GesturesApi file. To use, simply add the following import statement: from GesturesApi import GestureProcessor
GestureProcessor
, such as gp = GestureProcessor()
. You can optionally pass it a .txt
file which contains coordinates defining a gesture. If no file is specified, default gestures will be loaded (generation code can be found in defaultGesturesLoader.py
). Note: No checking is done for data integrity; it is assumed that the provided file meets the proper format specifications.bind()
method: gp.bind(index, fn)
. index
can either be the integer index of the gesture (in the order that the gestures were loaded) or a string containing the exact name of the gesture. fn
is a function object which takes no parameters; use closures as necessary (i.e. gp.bind(index, lambda: self.fn)
)gp.process()
. This will grab the next camera image and update the information inside gp
, including depth and palm center. If a gesture is detected, this will also call the action that was bound to it, and update gp.lastAction
with the name of the last gesture. Note: this call is expensive and will take anywhere between 2 to 5 ms on average, depending on the machine.gp.saveNext()
. This will add the next new gesture to the list of gestures with a random name. The random name is then set as gp.lastAction
, so the programmer can change it to something more useful if desired. Alternatively, the gesture will simply be the last one in gp.gestures
, and can be modified from there.gp.close()
. This will clean up created data and, importantly, release the camera. Failure to do so will result in the camera being active after the program appears to have exited, and will make it impossible for other applications to bind onto the camera (including new instances of the offending program.)The process()
loop can be summarized as follows: