MMM-Gestures is a head-less 3rd party module for MagicMirror that allows to control the Magic Mirror via gestures of two types of infrared sensors. Gestures can be up, down, left, right, far and close movements of the hand in front of a gesture sensor (APDS-9960) and present and away gestures in front of an IR distance sensor (GP2Y0A21YK).
In order to receive gesture events the following hardware is required:
The Arduino collects gesture events from gesture sensor APDS-9960 and distance from distance sensor GP2Y0A21YK (10-80cm), which are then forwarded on the serial port as text.
The circuit for an Arduino Uno:
In order to place the sensors below the mirror you can use "Hardware Sensor Module\Mirror Gesture Module.ai" to cut out wholes of the right size for the APDS-9960 and GP2Y0A21YK sensor. The file contains three elements, in the order they need to be "printed":
After wiring the sensors to the Arduino they can be glued to the first sheet from the back. Once dried, the other two to tree frames can be glued on the back of the first sheet.
In order to compile you'll need to copy the following libraries to Arduino's library folder. The libraries are also included in this git repository.
Once the Arduino is connected to both sensors and the software is setup it will send gesture events to the Raspberry Pi via USB. On the Raspberry Pi a Node.js programm will listen to these events and make them available via a WebSocket server.
modules folder and execute git clone https://github.com/thobach/MMM-Gestures.git.cd MMM-Gestures && npm installNote: If you use an older Raspberry Pi 2 or operating system (vs. 3), you may need to change the USB device name in https://github.com/thobach/MMM-Gestures/blob/master/node_helper.js#L104 from ACM to USB.
This module relies on electron-rebuild during the install process. If you run MagicMirror in a container (e.g. as described at )https://khassel.gitlab.io/magicmirror/), your host operating system may not recognize the electron version and give you the error An unhandled error occurred inside electron-rebuild Unable to find electron's version number of MagicMirror, either install it or specify an explicit version. To avoid this, you'll need to execute the install command from within the container:
docker exec -it mm bash, assuming your container is named mmCommunication between Raspberry Pi and Arduino happens via the serial port (USB). The node-helper.js app from this project creates the connection between the two systems (Raspberry Pi and Arduino), forwards gesture and distance events to the web user interface via the MagicMirror's built-in socket communication and also controls the HDMI display to save power if nobody has interacted with or stood in front of the mirror for 5 minutes.
In order to load the MMM-Gestures module you will need to add the following configuration to your config/config.js file.
modules: [
{
module: 'MMM-Gestures',
},
]The MMM-Gestures.js Magic Mirror module listens to socket events from the node-helper.js app and converts them to Magic Mirror-internal events that can be received via the built-in notificationReceived() function.
The received event has the following format:
Sample code for reacting to gesture events:
notificationReceived: function(notification, payload, sender) {
Log.info(this.name + " - received event");
if(notification == 'GESTURE'){
Log.info(this.name + " - received gesture");
var gesture = payload.gesture;
// actually RIGHT, because gesture sensor is built in upside down
if(gesture.startsWith('LEFT')){
Log.info(this.name + " - received right");
// adjust some internal representation ...
// update display
this.updateDom(100);
}
// actually LEFT, because gesture sensor is built in upside down
else if(gesture.startsWith('RIGHT')){
...
}
// gesture event that was neither LEFT or RIGHT received
else {
Log.info(this.name + " - received other: " + gesture);
}
}
},By default this module looks for the compliments module and only shows compliments when someone stands in front of the mirror. This is done by hiding the module by default, showing it when the "PRESENT" gesture is received and hiding the module again when the "AWAY" gesture is received. Further the user interface is reloaded when a FAR gesture is received, which can be useful user interface to testing purposes.
Available gestures:
Note: If the sensor is installed upside-down the events from the gesture sensor APDS-9960 are reversed, e.g. a left gesture would be received as RIGHT. This does not apply to FAR and CLOSE gestures.
You can find a video demonstration of how the sensor works at https://www.youtube.com/watch?v=OS36IdgpEIo. Note, this break-out board is not optimal for use in Magic Mirror since all components are soldered on the front side. On Aliexpress you find nicer breakout boards that have only the sensor on the front side and the other components soldered to the back.
The following modules react to gesture events from this module:
If you build a Gesture-enabled Magic Mirror module, please let me know or create a pull request and I'll link it here.