1j01 / tracky-mouse

Mouse control via head tracking, as a cross platform desktop app and JS library. eViacam alternative.
https://trackymouse.js.org/
MIT License
25 stars 4 forks source link

Trigger click with vocal gestures (pop, ssss, whistle) #38

Open 1j01 opened 1 year ago

1j01 commented 1 year ago

An idea seen in Polymouse (research software which looks hard to install, but is also intriguing.) Similar to facial gestures (#25), vocalizations would be a lot faster to execute, and more relaxed than dwell clicking, and can allow natural extending of the click by extending the sound (for dragging things, or painting, etc.)

Actually a hybrid with visual detection of facial gesture (#25) might be good. Make a popping sound with your lips to start a click, and close your mouth to stop it. That way, you can yawn without worrying about clicking (as opposed to an open mouth gesture), in theory. And it could require mouth movement combined with sound so that it's less likely to interpret some background noise as a click.

It would also be possible to use pitch and volume to vary brush parameters like thickness and opacity, if the program acted as a tablet driver, or if software wasn't so stupid, at a fundamental level. (I don't know how to make a driver! And software would have to be configured with an understanding that "pen pressure" means "pitch" or what have you. Stupid. Not stupid enough that it's not a good goal, but I hate the fact that that's the best we could hope for, without custom software for every application, y'know?) Or perhaps pitch could control speed of mouse movement with the head tracking... (Controlling something within the software like that would be easier to implement.)