Talon eye tracker breaks after Windows wake from sleep

[jamesmugford]

Restarting Talon fixes issue and eye tracker comes back online.

[zmcgohan2]

@ja-mesmugford Can you provide steps for reproducing this error? Minimally, I just turned on "control mouse", looked around, "talon sleep", mouse was still tracking my eyes surprisingly which might be its own bug***** (+ @lunixbochs), "talon wake", and had no issues continuing with Talon.

[lunixbochs]

"talon sleep" just sleeps speech, try "sleep all": https://github.com/knausj85/knausj_talon/blob/master/modes/wake_up.talon#L8

[zmcgohan2]

With "sleep all", behavior was expected. Mouse control was turned off, and then after turning back on I was able to control Talon (incl. turning back on control mouse) with no issues.

[zmcgohan2]

Still needs reproduction steps.

[jamesmugford]

Sorry, I wasn't very clear. The issue happens when the Windows operating system wakes from sleep.

I've tested this on a few machines, all from different manufacturers, different motherboards etc and all have this issue.

macOS seems to not have this issue.

[saidelike]

I have the same problem. On windows, after a sleep mode resume, and with talon already started, the eye tracking fails to get started. control mouse does not work. This is the output log:

2023-05-23 09:18:38 ERROR cb error topic="focus" cb=Calibration.on_canvas_focus
   11:                threading.py:937* # cron thread
   10:                threading.py:980* 
    9:                threading.py:917* 
    8:               talon\cron.py:156| 
    7: talon\scripting\dispatch.py:100| 
    6: -------------------------------# [stack splice]
    5: talon\scripting\dispatch.py:134| # 'focus' tracking:Calibration.on_canvas_focus()
    4:  talon_plugins\eye_mouse.py:428| self.stop()
    3:  talon_plugins\eye_mouse.py:453| self.cmd(tobii.CALIBRATE_STOP)
    2:  talon_plugins\eye_mouse.py:383| return self.tracker.cmd(*args)
    1:        talon\track\tobii.py:337| 
talon.track.tobii.EyeClosedErr: Eye Tracker timed out or disconnected

Restarting talon solves the problem.

[saidelike]

Another case:

2023-06-01 17:17:30 WARNING [watchdog] "Cron.interval.<locals>.call" @4.0s (stalled)
2023-06-02 09:02:54  INFO Deactivating Microphone
2023-06-02 09:02:54  INFO Activating Microphone: "Microphone (Shure Digital   )"
2023-06-02 09:02:55 WARNING [watchdog] "MicrophoneManager.on_wake" @2.0s
    9:              talon\windows\ui.py:425|
    8:      talon\scripting\dispatch.py:134| # 'screen_wake' main:MicrophoneManager.on_wake()
    7:      talon\plugins\microphone.py:78 |
    6:      talon\plugins\microphone.py:168|
    5:      talon\scripting\dispatch.py:134| # 'mic_change' main:SpeechSystem._set_microphone()
    4: talon\scripting\speech_system.py:199|
    3:                     talon\vad.py:122|
    2:                     talon\vad.py:49 |
    1:               talon\lib\cubeb.py:444|
2023-06-02 09:02:58 WARNING [watchdog] "<lambda>" @2.0s
    4:         talon\windows\ui.py:425|
    3: talon\scripting\dispatch.py:134| # 'screen_wake' talon_plugins.eye_mouse:<lambda>()
    2:  talon_plugins\eye_mouse.py:621| ui.register('screen_wake',  lambda e: ..
    1:        talon\track\tobii.py:330|
2023-06-02 09:03:00 WARNING [watchdog] "<lambda>" @4.0s (stalled)
2023-06-02 09:03:02 WARNING [watchdog] "<lambda>" @6.0s (stalled)
2023-06-02 09:03:04 WARNING [watchdog] "<lambda>" @8.0s (stalled)
2023-06-02 09:03:06  INFO Eye Tracker disappeared on wakeup, trying to reconnect
2023-06-02 09:03:06 WARNING [watchdog] "<lambda>" @10.0s
   13:         talon\windows\ui.py:425 |
   12: talon\scripting\dispatch.py:134 | # 'screen_wake' talon_plugins.eye_mouse:<lambda>()
   11:  talon_plugins\eye_mouse.py:621 | ui.register('screen_wake',  lambda e: ..
   10:        talon\track\tobii.py:349 |
    9:         logging\__init__.py:2097|
    8:         logging\__init__.py:1446|
    7:         logging\__init__.py:1589|
    6:         logging\__init__.py:1599|
    5:         logging\__init__.py:1661|
    4:         logging\__init__.py:952 |
    3:         logging\handlers.py:73  |
    2:         logging\handlers.py:189 |
    1:              genericpath.py:19  |
2023-06-02 09:03:06 ERROR Reconnecting to Eye Tracker
    6:         talon\windows\ui.py:425|
    5: talon\scripting\dispatch.py:134| # 'screen_wake' talon_plugins.eye_mouse:<lambda>()
    4:  talon_plugins\eye_mouse.py:621| ui.register('screen_wake',  lambda e: ..
    3:        talon\track\tobii.py:351| # [stack splice]
    2:        talon\track\tobii.py:265|
    1:        talon\track\tobii.py:310|
talon.track.tobii.EyeOpenErr: Eye Tracker open failed with error: error opening device
2023-06-02 09:03:06 ERROR cb error topic="screen_change" cb=on_screen_change
   12:                threading.py:937* # cron thread
   11:                threading.py:980*
   10:                threading.py:917*
    9:               talon\cron.py:156|
    8: talon\scripting\dispatch.py:100|
    7: talon\scripting\dispatch.py:134| # 'screen_update' main:proxy_screen_change()
    6:         talon\windows\ui.py:436|
    5: -------------------------------# [stack splice]
    4: talon\scripting\dispatch.py:134| # 'screen_change' talon_plugins.eye_mouse:on_screen_change()
    3:  talon_plugins\eye_mouse.py:638| tracker.display_setup(config.size_mm.x..
    2:        talon\track\tobii.py:408|
    1:        talon\track\tobii.py:337|
talon.track.tobii.EyeClosedErr: Eye Tracker timed out or disconnected

[saidelike]

For instance, it works fine on Linux after a wake from sleep:

2023-07-27 18:19:22.551    IO   user.talon_sleep: Put Talon to sleep
2023-07-27 18:19:22.551    IO ==============================
2023-07-28 07:57:26.434  INFO Deactivating Microphone
2023-07-28 07:57:26.438  INFO Activating Microphone: "X2u Adapter Analogue Stereo"
2023-07-28 07:57:28.450 WARNING [watchdog] "<lambda>" @2.0s
    4:           talon/linux/ui.py:395| 
    3: talon/scripting/dispatch.py:134| # 'screen_wake' talon_plugins.eye_mouse:<lambda>()
    2:  talon_plugins/eye_mouse.py:621| ui.register('screen_wake',  lambda e: ..
    1:        talon/track/tobii.py:331| 
2023-07-28 07:57:30.447 WARNING [watchdog] "<lambda>" @4.0s (stalled)
2023-07-28 07:57:31.446  INFO Eye Tracker disappeared on wakeup, trying to reconnect
2023-07-28 07:57:31.448  INFO Eye Tracker Firmware: 10008

[jamesmugford]

Did this just get fixed?

[jmegner]

I'm on Windows 10 and 11 and also have the problem that Talon does not work with the eye tracker after a Windows sleep. I currently know of two workarounds:

Workaround 1: Restart Talon

Restarting Talon makes the eye tracker work in Talon again. Unfortunately, you then have to wait for Talon to start up.

I have the talon command grizzly restart: user.talon_restart(), which uses the following python code inspired by Andreas Arvidsson:

import os
from talon import Module, ui

mod = Module()

@mod.action_class
class Actions:
    def talon_restart():
        """restart talon"""
        talon_app = ui.apps(pid=os.getpid())[0]
        print("Restarting:", talon_app)
        os.startfile(talon_app.exe)
        talon_app.quit()

    def talon_quit():
        """quit talon"""
        talon_app = ui.apps(pid=os.getpid())[0]
        print("Quitting:", talon_app)
        talon_app.quit()

I have suspected that the above does not truly kill the Talon process, and those processes live on and hold onto files and cause Talon update installations to fail (the fix being to reboot or go to Task Manager's Details tab and kill these processes). However, just now, after doing a lot of "grizzly restart" commands, I do not see any extra talon.exe processes.

So, I'm not sure if talon_app.quit() is actually problematic, but just in case: If anyone has a better way to get Talon to terminate itself, I would be grateful to hear it. Also, something like a Powershell script that can be linked to a global keyboard shortcut is almost as good.

Workaround 2: Unplug And Replug Eye Tracker

Unplugging and replugging the eye tracker seems to make the eye tracker work in Talon again. I haven't played around this very much, so am unaware of any consequences that are not immediately obvious.

[jmegner]

I had a convo with Ryan Hileman about the issue, and he said some things that are of general interest...

On Talon's functionality and his prior efforts:

• "talon does automatically connect to trackers after machine sleep...whatever is happening is more broken than that"
• "i've already tried multiple times to do the obvious fixes"
• "next idea is to reinitialize the usb stack from talon's perspective but that's super invasive"

On what a user can do in the meantime:

• "change the way your computer sleeps so it doesn't disconnect usb devices"
• "change your machine sleep state to be less deep"
• "the best thing you can do on your own is configure your machine to sleep less deeply, e.g. display sleep instead of machine sleep"
• On using a talon command to restart talon: "i do generally not like that"