How can I ensure that the `enigo.key` action is completed before proceeding with any subsequent task

[Mia-Zhang-Forever]

Describe your Question**

how to make sure the key event is completed?

Describe your Goal**

How can I ensure that the enigo.key action is completed before proceeding with any subsequent task

Environment (please complete the following information):**

• OS: [macOS]
• Rust [rustc 1.79.0 (129f3b996 2024-06-10)]
• Library Version [enigo = { git = "https://github.com/enigo-rs/enigo", rev = "a1614a6" }]
• Application: [Tauri]

Details

I'm creating a straightforward system tray application that interacts with the user's text selection and system clipboard. I have a simple thread that sends a keyboard event and then performs some additional tasks. it looks like this:

    handle_selection(selection_action);
    do_further_task();

In handle_selection, I use enigo.key to send a backspace key event for deleting the currently selected text.

enigo.key(Key::Backspace, Direction::Click).unwrap();

It works fine when the text is not within the same Tauri application (e.g. discord input box, apple notes, etc)

It behaves weirdly when the text selection is in a Tauri window.

what's wrong

In order to debug it, I swapped the implementation of do_further_task with a simple sleep.

    thread::yield_now();
    for _ in 0..3 {
        println!("waiting for 5 seconds");
        sleep(std::time::Duration::from_secs(1));
    }

Correct behavior (in apple notes):

The text selection is deleted, and then the sleep for loop begins.

Wrong behavior (in my tauri window):

The text selection is not deleted (the function is completed, and I have a log to prove it) until the for loop finishes. I even use the thread::yield_now() and it didn't help.

[pentamassiv]

It should just work. On macOS there currently are some sleeps too ensure the input gets entered on time. If it doesn't work for you, then there is a bug.

I did a bunch of work on enigo the last couple of months, so could you please try again with the most recent commit (bbaa7c7c370e83dcbd6c050b8ab885a845479ea7)? Hopefully that fixes the issue. If it doesn't, please let me know and we can try to investigate it

[pentamassiv]

If updating enigo doesn't help, it would also help me if you could provide me a link to your code

[Mia-Zhang-Forever]

If updating enigo doesn't help, it would also help me if you could provide me a link to your code

It doesn't work either. I've created a demo application for you and explained how to reproduce the bug (it might not be a bug, but how how event is handled?)

https://github.com/Mia-Zhang-Forever/enigo_demo/blob/main/README.md

[Mia-Zhang-Forever]

If updating enigo doesn't help, it would also help me if you could provide me a link to your code

It doesn't work either. I've created a demo application for you and explained how to reproduce the bug (it might not be a bug, but how how event is handled?)

https://github.com/Mia-Zhang-Forever/enigo_demo/blob/main/README.md

@pentamassiv Are you able to reproduce the issue?

[pentamassiv]

Unfortunately I don't own a Mac, so I can only test code for macOS when I borrow a device. I'll be able to do that in two weeks. In the meantime I will have a look at your code and check if there is anything that could cause this. I never used Tauri before, so it will likely take a few days

[Mia-Zhang-Forever]

Unfortunately I don't own a Mac, so I can only test code for macOS when I borrow a device. I'll be able to do that in two weeks. In the meantime I will have a look at your code and check if there is anything that could cause this. I never used Tauri before, so it will likely take a few days

I really appreciate that. In the meanwhile, I used cargo add get-selected-text as a workaround, It has a similar issue; however, it only occurs at the very first time of calling.

[Mia-Zhang-Forever]

Unfortunately I don't own a Mac, so I can only test code for macOS when I borrow a device. I'll be able to do that in two weeks. In the meantime I will have a look at your code and check if there is anything that could cause this. I never used Tauri before, so it will likely take a few days

By the way, Tauri should be able to run on Linux or Windows. You might try to compile it with the computer you have right now.

[pentamassiv]

I'll have a look now.

BTW, are you aware that there is an open issue regarding crashes with Tauri on macOS? I still have no idea why it is happening. Did you not experience any crashes?

[Mia-Zhang-Forever]

I'll have a look now.

BTW, are you aware that there is an open issue regarding crashes with Tauri on macOS? I still have no idea why it is happening. Did you not experience any crashes?

I haven't experienced the crash issue; I'm currently using Tauri 1.8.

[pentamassiv]

I tried to run the demo, but I got this error:

** (enigo-demo:43367): WARNING **: 23:43:46.072: webkit_settings_set_enable_offline_web_application_cache is deprecated and does nothing.
thread 'main' panicked at src/main.rs:49:18:
called `Result::unwrap()` on an `Err` value: GlobalShortcut(InvalidAccelerator("Unable to register global shortcut"))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I am using GNOME Wayland, so maybe that's the reason why. I think global shortcuts are not yet possible. I think I might just have to wait a bit.

Do you have this issue on all platforms or did you only test it on macOS?

[Mia-Zhang-Forever]

I tried to run the demo, but I got this error:

** (enigo-demo:43367): WARNING **: 23:43:46.072: webkit_settings_set_enable_offline_web_application_cache is deprecated and does nothing.
thread 'main' panicked at src/main.rs:49:18:
called `Result::unwrap()` on an `Err` value: GlobalShortcut(InvalidAccelerator("Unable to register global shortcut"))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I am using GNOME Wayland, so maybe that's the reason why. I think global shortcuts are not yet possible. I think I might just have to wait a bit.

Do you have this issue on all platforms or did you only test it on macOS?

@pentamassiv Right now, I have only tested it on macOS.

[Mia-Zhang-Forever]

I tried to run the demo, but I got this error:

** (enigo-demo:43367): WARNING **: 23:43:46.072: webkit_settings_set_enable_offline_web_application_cache is deprecated and does nothing.
thread 'main' panicked at src/main.rs:49:18:
called `Result::unwrap()` on an `Err` value: GlobalShortcut(InvalidAccelerator("Unable to register global shortcut"))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I am using GNOME Wayland, so maybe that's the reason why. I think global shortcuts are not yet possible. I think I might just have to wait a bit.

Do you have this issue on all platforms or did you only test it on macOS?

any update on this, my friend

[pentamassiv]

No yet, sorry. I am swamped with work. I'll try to look at it tomorrow. The good news is that I will be able to test the macOS parts anytime I need to now

[Mia-Zhang-Forever]

No yet, sorry. I am swamped with work. I'll try to look at it tomorrow. The good news is that I will be able to test the macOS parts anytime I need to now

Thanks, my friend. I have an issue with the Windows system too. I just rented one, and I will do some tests and let you know.

[pentamassiv]

Alright, that's nice.

I fixed an issue with entering newlines on Windows yesterday, so make sure to use the current version :-)

[pentamassiv]

So I looked at the issue and your demo. I also created my own simplified version of it with tauri 2.0 here. I experienced the same issue but I don't think there is anything to do on my side. The keyboard events get sent to the OS by enigo basically put into a queue. There is no way for enigo to know if the events were processed on macOS.

I suggest looking into the event handling in tauri. My guess is that there is a RunLoop that handles events. If you enter a keyboard shortcut, an event gets created for the shortcut and added to the RunLoop. Once the RunLoop processes that event, it will execute the code you associated with it (get_context()). That code simulates keyboard input, so these events will get added to the queue of the OS and eventually passed to the queue of events the RunLoop handles. Those keyboard events will not be handled by tauri until the RunLoop processes them. It does not interrupt your code. The clipboard_manager gets the text that is stored in the clipboard at that moment. The problem is that the keyboard events to copy the highlighted code were not yet processed so the text you copied at some point in time before is returned instead.

You somehow need to allow tauri to process the events in the RunLoop before you get the text from the clipboard. Today's the first time I used tauri, so I don't know how to do that. Maybe looking into RunEvent will help?

If you find out how to fix the problem or if my suspicion was wrong, please let me know. Tauri seems to be quite popular so a simple example on how to use enigo with Tauri would help a lot.

[pentamassiv]

PS: It is advisable to keep the Enigo struct around and reuse it instead of building a new one each time

[Mia-Zhang-Forever]

Thanks for the detailed analysis and for creating a simplified version to test the issue.

Your hypothesis about the RunLoop makes a lot of sense. I will try to investigate more on my side.