enigo-rs / enigo

Cross platform input simulation in Rust
MIT License
886 stars 95 forks source link

Can't get enigo working in Tauri app on MacOS #174

Open garrrikkotua opened 1 year ago

garrrikkotua commented 1 year ago

Describe your Question Can't get enigo working in Tauri app on macOS.

Describe your Goal I want to have enigo typing (key_sequence function) in Tauri app, but it doesn't work at all. I want it to type in the app itself and other apps too. This functionality works great on Windows.

Environment (please complete the following information):

Note I use set_activation_policy(tauri::ActivationPolicy::Accessory) in Tauri, so the app behaves like a spotlight window. Maybe it prevents some functionality

pentamassiv commented 1 year ago

We already have an issue for Tauri: https://github.com/enigo-rs/enigo/issues/153 Is your problem different?

garrrikkotua commented 1 year ago

Yeah, I think it is related. Basically key_sequence is a sequence of key_clicks, right?

But in my case the app doesn't crush, just nothing happens. I thought this might be due to some permissions on macOS, etc

pentamassiv commented 1 year ago

key_sequence is not just a sequence of key_click. key_sequence enters the whole text at the same time. It should probably be renamed to text_input or something like that. Could you please try key_click and see if it panics?

garrrikkotua commented 1 year ago

Yeah I will try later when I have mac :) Perhaps tommorrow

pentamassiv commented 8 months ago

I am not familiar with Tauri, but I had to grant permissions in a popup. Did you grant the application the Accessibility permissions? https://support.apple.com/guide/mac-help/allow-accessibility-apps-to-access-your-mac-mh43185/mac

pentamassiv commented 7 months ago

Is this still an issue or can I close it?

skyslide22 commented 5 months ago

i have the same issue, enigo does not fire anything MacOS Sonoma 14.1.2 with M3 Enigo 0.1.3 & 2.0rc2 tested

#[tauri::command]
async fn switch_space(left: bool) {
    let mut enigo= Enigo::new(&Settings::default()).unwrap();
    enigo.key(Key::Control, Direction::Press);
    if left {
        enigo.key(Key::LeftArrow, Direction::Click);
    } else {
        enigo.key(Key::RightArrow, Direction::Click);
    }
    enigo.key(Key::Control, Direction::Release);
    println!("switch space left = {left}") // this is printed but key did not fire at all
}
A-Shleifman commented 3 months ago

I'm writing my first rust script now, so I have no idea what I'm doing, but maybe the problem is a level lower. My script is not using Tauri, but it's using tao, which is what Tauri runs on top of from what I've read.

use global_hotkey::{ hotkey::{HotKey, Code}, GlobalHotKeyEvent, GlobalHotKeyManager, HotKeyState };
use tao::event_loop::{ ControlFlow, EventLoopBuilder };
use enigo::{ Enigo, Keyboard, Settings };

fn main() {
    let event_loop = EventLoopBuilder::new().build();
    let mut enigo = Enigo::new(&Settings::default()).unwrap();
    let hotkey_manager = GlobalHotKeyManager::new().unwrap();

    enigo.text("Launched!").unwrap(); // 👈 this is working as expected

    let hotkey = HotKey::new(None, Code::F12);

    hotkey_manager.register(hotkey).unwrap();

    let global_hotkey_channel = GlobalHotKeyEvent::receiver();

    event_loop.run(move |_event, _, control_flow| {
        *control_flow = ControlFlow::Wait;

        if let Ok(event) = global_hotkey_channel.try_recv() {
            if hotkey.id() == event.id && event.state == HotKeyState::Pressed {
                println!("F12 Pressed!");

                enigo.text("input").unwrap(); // 👈 this doesn't do anything
            }
        }
    })
}

In this script enigo.text works outside tao's event_loop, but not inside. This could make hunting for the problem a bit easier.

thewh1teagle commented 1 week ago

I also facing the same issue with tauri app. enigo works great outside of tauri, also it works with tauri in dev mode. but once I install the app and launch through applications then enigo keys doesn't work. Production log:

tauri installed app logs ```console [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::app_delegate] Triggered `applicationDidFinishLaunching` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window] Creating new window [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window] Locked shared state in `set_fullscreen` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window] Unlocked shared state in `set_fullscreen` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidBecomeKey:` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidBecomeKey:` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow` [2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::app_delegate] Completed `applicationDidFinishLaunching` [2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidResignKey:` [2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidResignKey:` [2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::app_delegate] Triggered `applicationSupportsSecureRestorableState` [2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::app_delegate] Completed `applicationSupportsSecureRestorableState` [2024-05-06T23:06:16Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidBecomeKey:` [2024-05-06T23:06:16Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidBecomeKey:` [2024-05-06T23:06:18Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidResignKey:` [2024-05-06T23:06:18Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidResignKey:` [2024-05-06T23:06:23Z DEBUG enigo] using default settings [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:23Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:23Z DEBUG enigo] using default settings [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:23Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:25Z DEBUG enigo] using default settings [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:25Z DEBUG mobslide::cmd] Pressing VOL_DN [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click) [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:25Z DEBUG enigo] using default settings [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:25Z DEBUG mobslide::cmd] Pressing VOL_DN [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click) [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key [2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:31Z DEBUG enigo] using default settings [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:31Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:31Z DEBUG enigo] using default settings [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:31Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:34Z DEBUG enigo] using default settings [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:34Z DEBUG mobslide::cmd] Pressing VOL_DN [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click) [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:06:34Z DEBUG enigo] using default settings [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:06:34Z DEBUG mobslide::cmd] Pressing VOL_DN [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click) [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key [2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] released all held keys ```
tauri dev mode logs (works) ```console Finished dev [unoptimized + debuginfo] target(s) in 3.65s [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::app_delegate] Triggered `applicationDidFinishLaunching` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window] Creating new window [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window] Locked shared state in `set_fullscreen` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window] Unlocked shared state in `set_fullscreen` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::app_delegate] Completed `applicationDidFinishLaunching` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidBecomeKey:` [2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidBecomeKey:` 🌼 daisyUI 4.10.5 ├─ ✔︎ 2 themes added https://daisyui.com/docs/themes ╰─ ★ Star daisyUI on GitHub https://github.com/saadeghi/daisyui [2024-05-06T23:09:42Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidResignKey:` [2024-05-06T23:09:42Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidResignKey:` [2024-05-06T23:09:44Z DEBUG enigo] using default settings [2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:09:44Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:09:46Z DEBUG enigo] using default settings [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:09:46Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:09:46Z DEBUG enigo] using default settings [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:09:46Z DEBUG mobslide::cmd] Pressing VOL_UP [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click) [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key [2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:09:47Z DEBUG enigo] using default settings [2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:09:47Z DEBUG mobslide::cmd] Pressing VOL_DN [2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click) [2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key [2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] released all held keys [2024-05-06T23:09:48Z DEBUG enigo] using default settings [2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] connection established on macOS [2024-05-06T23:09:48Z DEBUG mobslide::cmd] Pressing VOL_DN [2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click) [2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key [2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] released all held keys ```
cargo.toml ```toml enigo = { version = "0.2.0" } ```

Update

Found the cause, it's permission error

For manage that permission you need to open settings -> accessibility

screenshot

Then click the + button to add your app from application and allow it to control the computer If you have already the app it's possibly the dev mode app, remove it and add again the installed one. Once you remove it and reopen the app it should open the following prompt to ask for the permission:

screenshot

When the permission allowed enigo should work. By the way, it will be nice if enigo can detect that the app doesn't have that required permission and show error

https://stackoverflow.com/questions/6933510

Checking it is simple as

permission.rs ```rust use std::{error::Error, ptr}; use accessibility_sys::{kAXTrustedCheckOptionPrompt, AXIsProcessTrustedWithOptions}; use core_foundation_sys::dictionary::{CFDictionaryAddValue, CFDictionaryCreateMutable}; use core_foundation_sys::base::{CFRelease, TCFTypeRef}; use core_foundation_sys::number::{kCFBooleanFalse, kCFBooleanTrue}; fn check_accessibility(ask_if_not_allowed: bool) -> Result> { let is_allowed; unsafe { let options = CFDictionaryCreateMutable(ptr::null_mut(), 0, std::ptr::null(), std::ptr::null()); let key = kAXTrustedCheckOptionPrompt; let value = if ask_if_not_allowed {kCFBooleanTrue} else {kCFBooleanFalse}; if !options.is_null() { CFDictionaryAddValue( options, key.as_void_ptr(), value.as_void_ptr(), ); is_allowed = AXIsProcessTrustedWithOptions(options); CFRelease(options as *const _); } else { return Err("options is null".into()); } } Ok(is_allowed) } fn main() { let is_allowed = check_accessibility(true).unwrap(); println!("Accessibility permission enabled: {}", is_allowed); } ```