I can't move the cursor with my head

[MetaRevenge]

It's an amazing project, keep up the good work.

At the moment I can't move the cursor with my head, Face Control works fine, I can use the facial gestures without problems but the cursor doesn't respond to my head movements, I use Windows 11 Pro and a Microsoft Livecam Studio.

Run as administrator doesn't work.

[acidcoke]

take a look at #4

[MetaRevenge]

take a look at #4

Thanks, but running it as Administrator didn't work for me.

[Little-Den]

It helped me to unzip 'base_library.zip' in directory of "project_gameface" directly to itself, also i run 'as administrator'

[MetaRevenge]

It helped me to unzip 'base_library.zip' in directory of "project_gameface" directly to itself, also i run 'as administrator'

Thanks, but even though I followed your steps it didn't work.

[afgs10]

Same problem. Running windows 11 on a thinkpad x395. currently i am unable to game, and found this program appealing. I tried running as admin, unziping the library mentioned in this issue, changing several settings in the programs default profile and used two different webcams, all unsuccessful. Cannot make the mouse move with my head, even though my head and its direction is being tracked, and that tracking shown on the bottom left preview of the cam. The only time i can move the mouse is by setting a gesture to center the mouse on the screen. Then, and only then, does the mouse move, but only to the center of the screen... I hope this project wasn't just some publicity stunt by google, and continues to get updated :/

[githubpsyche]

I'm interested in using this feature too and have started some debugging via the Python app. I've found an odd hack through breakpoint toggling that gets things working correctly but haven't identified the change in the codebase that will actually address this issue.

The sequence though is that in VSCode, I:

• Toggle a breakpoint at line 49 of task_killer.py. This is the line right after the MouseController is first initialized by the app.
• Run the app through the debugger up to this breakpoint
• Go to the "Call Stack" and switch to ThreadPoolExecutor-1_0 and then select under that "mouse_controller.py".
• Press play/allow the code to run from there.

For whatever reason, the app works as it's supposed to do at this point.

Best wishes to the person whose job it is to figure out why

[oneswitch]

I had the same issue: https://switchgaming.blogspot.com/2023/05/project-gameface.html

I'm using Windows 10 and a Logitech 1080p camera. Whether in admin mode or not, the mouse pointer never moves. I can see a tracking dot between my nose, and I have the system enabled. Gestures work very well mapped to mouse buttons or key strokes however.

[Sitzstreiker]

Ich hab das selbe Problem und würde aufgrund meiner Behinderung die Software zu gerne testen...

[Gimpygod]

Same problem. Running windows 11 on a thinkpad x395. currently i am unable to game, and found this program appealing. I tried running as admin, unziping the library mentioned in this issue, changing several settings in the programs default profile and used two different webcams, all unsuccessful. Cannot make the mouse move with my head, even though my head and its direction is being tracked, and that tracking shown on the bottom left preview of the cam. The only time i can move the mouse is by setting a gesture to center the mouse on the screen. Then, and only then, does the mouse move, but only to the center of the screen... I hope this project wasn't just some publicity stunt by google, and continues to get updated :/

I am Lance aka GimpyG the fella in the Google IO presentation for Gameface, the good looking guy…who also is quad. The idea for Gameface was mine they agreed to help, I was in charge of what functions were included and determined if solutions were viable or had to be reworked. This project is a dream of mine to create a free Headmouse that allowed multiple switch inputs accessed by face only. Having said that, holy smokes I want to get this working for you! Have you adjusted the sensitivity bars for up, down, left & right? Are you able to see a little tracking dot on your face in the webcam preview window? We gotta get you smiting enemies asap!

[afgs10]

GimpyG the fella in the Google IO presentation for Gameface, the good looking guy…who also is quad. The idea for Gam

Thank you for the Reply Gimpy. I have adjusted the sliders up and down, left and right all of them tbh trying to get it to work. I am able to see the tracking dot on the bottom left webcam preview and it seems to be tracking my head movements quite well actually, it's just that it does not translate to any cursor movement.

As an aside, i see this concept, of using the movements of the head ( and expressions) as an input method akin to a mouse, as groundbreaking. Following a diagnosis involving my joints i began trying alternative input methods that would let me control the computer without pain. By far, the cheapest, most accessible, way that i found to do this would be using hardware i already had (webcam) and moving the mouse to where my head (eyes) were pointing at. I've tried this concept with many other programs ( eviacam and other headtracking software) and there were always issues that prevented me from using these reliably. The few moments i got it to kinda work though, were magical, not only did i regain my ability to quickly move the cursor without putting extra strain (and in my case pain) on my hands, but it actually sort of felt like a superior input method for regular work when compared to the mouse. i could read pdf's and have the cursor follow along as i read (as long as i moved my head instead of my eyes to read) even selecting text to underline. Plus, in conjunction with the excellent voice recognition systems we have for dictation, it truly felt like a futuristic, hands-free, accessible computer, not only for us with some physical limitations but maybe even for those without disabilities. TL;DR: i consider this input method as a lifeline to many in the disabled community and it could be so good it could even improve non-disabled people's computer use. I'm thankful for the work done so far, and your motivation to get this far, and excited by the contributions and interest that this github page has already gotten.

[Gimpygod]

“. TL;DR: i consider this input method as a lifeline to many in the disabled community and it could be so good it could even improve non-disabled people's computer use. I'm thankful for the work done so far, and your motivation to get this far, and excited by the contributions and interest that this github page has already gotten.”

thanks so much for your kind words and support! Okay let’s get this sorted. If you have been reading other posts you are not the only person having this issue but it seems rare. I think you said the keyboard binds work fine via face input and you see the mapping dot so you are being seen. Are you able to left or right click with facial gestures? If not maybe some other program is interfering with Gameface like the accessibility apps you mentioned? I am going off of experience chasing ways to interact with the computer in the past not Gameface related. I wish this issue would have happened to me so my Google buddies would have seen it, those people are ridiculously smart & knowledgeable. I look forward to hearing back from you.

[afgs10]

Yes! i am able to right click and left click with gestures. Center the mouse with gesture also works,(which is weird since it has to move the cursor to do it and might suggest the problem could be easy to solve). This weekend i will format my laptop just so i can be sure the problem isn't some leftover program or setting from other software. If that does not solve it, i will probably try to run it in another computer and see how it runs.

“. TL;DR: i consider this input method as a lifeline to many in the disabled community and it could be so good it could even improve non-disabled people's computer use. I'm thankful for the work done so far, and your motivation to get this far, and excited by the contributions and interest that this github page has already gotten.”

thanks so much for your kind words and support! Okay let’s get this sorted. If you have been reading other posts you are not the only person having this issue but it seems rare. I think you said the keyboard binds work fine via face input and you see the mapping dot so you are being seen. Are you able to left or right click with facial gestures? If not maybe some other program is interfering with Gameface like the accessibility apps you mentioned? I am going off of experience chasing ways to interact with the computer in the past not Gameface related. I wish this issue would have happened to me so my Google buddies would have seen it, those people are ridiculously smart & knowledgeable. I look forward to hearing back from you.

[acidcoke]

Center the mouse with gesture also works,(which is weird since it has to move the cursor to do it and might suggest the problem could be easy to solve).

Yeah, to me this indicates that it is more of a recognition problem.

Are you using glasses or anything that might be interfering with the recognition of your face? Also, how are you moving your head exactly? Do you turn your head or do just move it parallel to the plane of the monitor?

[githubpsyche]

No, I'm pretty sure it's not a recognition problem but rather a problem with the code.

My work finding that quirk in VSCode's debugger can unlock normal cursor control without changes to sensitivity settings seems to have revealed that the head-based cursor control process runs separately from the gesture-cue process but is normally killed during app startup. Toggling breakpoints as I specified above seems to keep the dying from happening.

IMO efforts to address this issue should focus on figuring out why this process dies and how to keep it from happening. I'd agree that it probably has its roots in some interaction w/ other Windows OS processes. The "Run as Administrator" fix is a useful clue supporting this idea. Correspondingly bet that no one who has this issue is using another OS.

[acidcoke]

Ah sorry I forgot about that.

[Gimpygod]

I have only used Gameface on a Lenovo laptop toaster and the Star Forge Creator Elite desktop that is built from the absolute top end components. Both were using a Logitech 940c webcam & haven’t experienced this problem, sigh. If anyone in the community wants to take on this issue we would be grateful, wish the program would generate a log for diagnostic purposes.

[Gimpygod]

“> No, I'm pretty sure it's not a recognition problem but rather a problem with the code.

My work finding that quirk in VSCode's debugger can unlock normal cursor control without changes to sensitivity settings seems to have revealed that the head-based cursor control process runs separately from the gesture-cue process”

Google Product manager for Gameface, Avneet Singh, wrote a blog post that will drop this week explaining the different aspects of MediaPipe being used to make the magic happen. I am extremely ignorant about the programming but I think you are correct that two separate processes for facial recognition are used to control the cursor movement and discern face gestures. Were you able to use a gesture to left or right click? The centering function is basically a macro so fits under keyboard control rather than mouse so it working doesn’t inform us about mouse functions. I want to say how much I appreciate your input and working towards making Gameface better.

[afgs10]

“> No, I'm pretty sure it's not a recognition problem but rather a problem with the code.

My work finding that quirk in VSCode's debugger can unlock normal cursor control without changes to sensitivity settings seems to have revealed that the head-based cursor control process runs separately from the gesture-cue process”

Google Product manager for Gameface, Avneet Singh, wrote a blog post that will drop this week explaining the different aspects of MediaPipe being used to make the magic happen. I am extremely ignorant about the programming but I think you are correct that two separate processes for facial recognition are used to control the cursor movement and discern face gestures. Were you able to use a gesture to left or right click? The centering function is basically a macro so fits under keyboard control rather than mouse so it working doesn’t inform us about mouse functions. I want to say how much I appreciate your input and working towards making Gameface better.

Following up on my previous message in the issue, i have now installed the program in another, completely unrelated laptop and found the same issue. Can't move the mouse, but everything else is working

I was able to use gestures to left and right click on the two pcs i've tried. Only difficulty is with moving the cursor

[code0nwheels]

@afgs10 try (or have someone) move a regular mouse while trying to use gf. I use a Quha Zono to move the mouse and when I pause it, gf does nothing but as soon as I unpause, gf seems to move the cursor. The cursor becomes extremely erratic until I disable gf - like the two are fighting each other.

Actually after typing that out, do you have a regular mouse plugged in? You know how with Windows 10 (possibly 11 too), the cursor disappears when no mouse is plugged in? I wonder if that has something to do with this.

[afgs10]

!!!!! somehow it worked!!!! thank you so much ! following @code0nwheels instructions i unplugged all my usb peripherals, turned on gameface and it is now working ! Not only that after i plug the usb hub back in it keeps on working! @Gimpygod i think @code0nwheels is on to something here. this mouse pointer bug thing described in this issue might be, somehow, usb peripheral related

[afgs10]

@MetaRevenge @oneswitch i urge you to try and unplug any usb devices and only after that turning on project gameface. If that doesn't work try and just try different combinations of plugging on/off usb devices and turning gameface on/off

[Gimpygod]

I am so happy 😁!!! Been wracking my brain trying to figure out why the dang thing was not working for you and @code0nwheels solved the issue! I wrote that a conflict was my guess but when you switched to the laptop and still experienced the same problem I was stumped 🤔. @code0nwheels you are a legend! Have fun playing with Gameface and spread the word.

[githubpsyche]

Awesome! This workaround is much easier than the debugger breakpoint thing and works on my system.

I think that in order to resolve this issue fully, we still need to understand why these steps are needed and find a fix to the codebase that either makes them unnecessary or at least more consistently lets users know that they may need to undergo them.

[willwade]

If you read the pyautogui issues there are a quite a lot of similar issues (pyautogui is what gameface is using under the hood for mouse). See https://github.com/asweigart/pyautogui/issues?q=is%3Aissue+is%3Aopen+mouse

Now pyautogui doesnt have any dependencies so the buck stops with that.

Out of interest if you run this code do you see the mouse still or is it gone? (NB: the code is actually this (It will loop - you might have to force quit it to stop it moving = press alt+f4 on the console window that appears. I've zipped it up too so unzip then run the exe)

(NB: I'm not entirely sure about the logic/thinking of using pydirectinput for keys and pyautogui for mouse - I'm not sure what exactly pydirectinput does over other libraries say pynput. I bet it's the smoothing and stuff.. but its a pain. #2 )

[oneswitch]

@MetaRevenge @oneswitch i urge you to try and unplug any usb devices and only after that turning on project gameface. If that doesn't work try and just try different combinations of plugging on/off usb devices and turning gameface on/off

Thanks @afgs10. Unplugging my Elgato HD60 capture device before starting Gameface gives me the intended mouse pointer control from the camera. Additionally, once Gameface is running, I can then reconnect the capture card, and Gameface mouse control will stay working. Not ideal, but progress and hopefully helpful for coders.

@willwade - Yes, I could see the mouse pointer. It placed the pointer into 4 positions one after the other, in a box like arrangement.

[willwade]

thanks @oneswitch i think the answer maybe to swap out pyautogui in the code for pynput - re: issue https://github.com/google/project-gameface/issues/9 - maybe first step is just to do this for mouse and ignore keys for now. I'll try and do this in my PR https://github.com/google/project-gameface/pull/11

[avneetgoogle]

Folks, thank you so much for reporting this issue. We'll look into this problem and try and see if we can find a code fix for this promptly

[ephsf]

Just in case anyone is still having issues with this - this drove me crazy for a while and then after some investigating, it appears that Windows power management is causing the issue. Under these circumstances, it will always work (the head tracking).

1. Run your laptop on power (if you are on a desktop - perfect).
2. Do not change the power setup (ie, don't change what USB peripherals are plugged in).
3. Run the app as an administrator
4. Then open gameface and turn on face tracking
5. Now the actions and tracking will work.

If I run my laptops on battery (both Windows 11 and Windows 10 versions - one is a high powered Razer Blade and the other a work productivity laptop), or I change USB peripherals or I run the app without running as admin, then I get the actions but no head tracking.

I hope this helps someone.