Control and move the pointer, and type, using head movements and facial gestures.
Paths have forward slash separators to be compatible with PowerShell and other command line interpreters.
Install Python 3.10 for Windows, from here for example.
python.org/downloads/release/python-31011/
Create a Python virtual environment (venv).
cd /path/where/you/cloned/FaceCommander
python -m venv venv
Install the required PIP modules into the venv.
./venv/Scripts/python.exe -m pip install --upgrade pip
./venv/Scripts/python.exe -m pip install -r ./requirements.txt
Run the application.
./venv/Scripts/python.exe face_commander.py
In case of difficulty or for details, see the Developer guide.
camera_id | Default camera index on your machine. |
tracking_vert_idxs | Tracking points for controlling cursor (see) |
spd_up | Cursor speed in the upward direction |
spd_down | Cursor speed in downward direction |
spd_left | Cursor speed in left direction |
spd_right | Cursor speed in right direction |
pointer_smooth | Amount of cursor smoothness |
shape_smooth | Reduces the flickering of the action |
tick_interval_ms | interval between each tick of the pipeline in milliseconds |
hold_trigger_ms | Hold action trigger delay in milliseconds |
rapid_fire_interval_ms | interval between each activation of the action in milliseconds |
auto_play | Automatically begin playing when you launch the program |
enable | Enable cursor control |
mouse_acceleration | Make the cursor move faster when the head moves quickly |
use_transformation_matrix | Control cursor using head direction (tracking_vert_idxs will be ignored) |
The config parameters for keybinding configuration are in this structure.
gesture_name: [device_name, action_name, threshold, trigger_type]
gesture_name | Face expression name, see the list |
device_name | "meta", "mouse", or "keyboard" |
action_name | name of the action e.g. "left" for mouse. e.g. "ctrl" for keyboard e.g. "pause" for meta |
threshold | The action trigger threshold has values ranging from 0.0 to 1.0. |
trigger_type | "single" for a single trigger "hold" for ongoing action. "dynamic" for a mixture of single and hold. It first acts like single and after passing the amount of miliseconds from hold_trigger_ms like hold. Note: this is the default behaviour for mouse buttons "toggle" to switch an action on and off "rapid" trigger an action every "rapid_fire_interval_ms" |
Blink graphics in the user interface are based on Eye icons created by Kiranshastry - Flaticon.
MediaPipe Face Landmark Detection API Task Guide
MediaPipe BlazeFace Model Card
MediaPipe FaceMesh Model Card
Mediapipe Blendshape V2 Model Card