search

zkxs / simple-crosshair-overlay

A simple and performant crosshair overlay
GNU General Public License v3.0
15 stars 3 forks source link

Linux Support #6

Open zkxs opened 11 months ago

zkxs commented 11 months ago

Hurdles:

rpdelaney commented 8 months ago

Subscribed + I would be happy to help test things. Using sway on wayland, lots of proton steam games and others.

zkxs commented 8 months ago

Thanks for the interest! The main barrier is that the interaction between the tray-icon library, GTK, and which threads things run on is pretty complicated (relative to Win/Mac at least) and the example code doesn't make it clear how to do things like receive button events and read/write checkbox state.

Anyways, I don't actually have this is a state where there's anything to test. The Linux build doesn't compile right now.

I have exactly one idea left to try which is completely ignoring the tray-icon example code's claim that the tray icon must be built on the GTK thread. I'll take a look at this again and see if I can at least get something compiled for you to test.

zkxs commented 8 months ago

@rpdelaney I've finally got a Linux build if you want to give it a spin. While it's my best-effort attempt it's completely untested, so who knows if it'll work?

The build. This is just an ELF inside of a zip file, because that's how github packages workflow artifacts. If you'd rather build it yourself, you can build from the linux-thread-switcharoo branch.

If it's totally broken I suppose I'll have to stop being lazy and install a Linux VM so I can iterate on it...

rpdelaney commented 8 months ago 
./simple-crosshair-overlay: error while loading shared libraries: libxdo.so.3: cannot open shared object file: No such file or directory

I might fix this by installing xdotool, but libxdo isn't gonna work on wayland though.

zkxs commented 8 months ago

I haven't used Linux on desktop since around 2012 (oh god that's a decade ago) so I'm definitely not the most up-to-date on how all this works today. We'll see what I can figure out...

Hmm, libxdo is a transitive dependency of the tray-icon library:

simple-crosshair-overlay v1.0.1
└── tray-icon v0.9.0
    └── muda v0.9.4
        └── libxdo v0.6.0
            └── libxdo-sys v0.11.0

I can't find anything except this comment referencing Wayland, and all it says is that Wayland "should work". Their readme also makes note of why they have a libxdo dependency... and it's all clipboard access features I'm not actually using. It looks like tray-icon lets me opt out of the libxdo dependency, so I'll give it a try.

Well, it compiles without libxdo so that's a good sign. Here's the new build zip file. And as before if you want to build it yourself, the linux-thread-switcharoo branch is still the one to use.

rpdelaney commented 8 months ago

Now it runs, but there's no observable behavior. 🤔 Perhaps this is a result of integration with the system tray? "System tray" is not a well-defined concept in the *NIX ecosystem...

zkxs commented 8 months ago

Even if the tray is broken, by default it's supposed to create a 16x16 pixel window in the center of your primary monitor. The window has a bunch of weird settings on it:

If the program is crashing it ought to give you an error message if you launch it from a terminal. It's also possible that it's hanging on startup, though. I did a weird threading thing to try and make it wait for the GTK thread to start, which shouldn't be deadlocking, but it might be.

zkxs commented 7 months ago

I've looked into this some more, and it turns out that the device-query library I'm using to get keyboard input is incompatible with Wayland, so a complete rewrite of the keyboard input system would be needed to get this working.

edit: my source

zkxs commented 7 months ago

After fixing the hang on startup in 351152529ddd082f577893d4f40eadb267fefe05 which was due to me not using condvar correctly, unfortunately I get a crash:

spawned GTK background thread
acquired GTK lock
waiting for GTK init signal
starting GTK background thread
GTK init complete
GTK init signal sent. Starting GTK main loop.
GTK startup complete
thread 'main' panicked at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gtk-0.18.1/src/auto/menu.rs:29:9:
GTK may only be used from the main thread.
stack backtrace:
   0: rust_begin_unwind
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
   1: core::panicking::panic_fmt
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
   2: gtk::auto::menu::Menu::new
             at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gtk-0.18.1/src/auto/menu.rs:29:9
   3: muda::platform_impl::platform::Menu::gtk_context_menu
             at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/muda-0.10.0/src/platform_impl/gtk/mod.rs:374:40
   4: <muda::menu::Menu as muda::ContextMenu>::gtk_context_menu
             at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/muda-0.10.0/src/menu.rs:354:9
   5: tray_icon::platform_impl::platform::TrayIcon::new
             at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tray-icon-0.10.0/src/platform_impl/gtk/mod.rs:38:37
   6: tray_icon::TrayIcon::with_id
             at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tray-icon-0.10.0/src/lib.rs:315:40
   7: tray_icon::TrayIconBuilder::build
             at /home/michael/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tray-icon-0.10.0/src/lib.rs:278:9
   8: simple_crosshair_overlay::main
             at ./src/main.rs:123:9
   9: core::ops::function::FnOnce::call_once
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

What's happening here is tray_icon_builder.build().unwrap() can only be called on the GTK thread, between calling gtk::init() and gtk::main(). I was afraid we'd end up back here, because I'm out of ideas as to how to unravel tray-icon's Linux threading requirements. Nothing left to do but ask for help: https://github.com/tauri-apps/tray-icon/issues/99

StellarHarbour commented 3 months ago

So there's still no way to make it work on Linux?

zkxs commented 3 months ago

The hoops I have to jump through on Linux to use the tray-icon crate are pretty extreme compared to Windows (and probably Mac) where it just kinda works how I need. Even if I totally rewrite how I talk with the tray icon state internally (which I might need to do for #51 regardless), the next hurdle is that the method I use to capture keypresses while my application window isn't in the foreground doesn't work on Wayland. I'm not really sure how to deal with that, because none of my options are good:

  1. ditch keyboard input entirely (good luck adjusting the crosshair position in a way that doesn't suck)
  2. move all keyboard input to a foreground-window-only method (but then how do I get the window focused in the first place?)
  3. create completely custom background input capture code just for Wayland, that'd have to run as root. To my knowledge no one has made a library that does this yet. EDIT: https://github.com/helium18/keylogger/ exists
StellarHarbour commented 3 months ago

Some guys recommended to use MangoHUD as crosshair overlay, and my first thought was that MangoHUD is overlay by itself for sure, can't you use approaches they used?

rpdelaney commented 3 months ago

Here's my mangohud config which I am currently using to project a crosshair on various games. 1080p display.

I have no idea if the games anyone else is playing have anti-cheat software that will ban you if you do this. I have no idea if it's considered unethical in the multiplayer games you play. All risk and liability for using this is with you. MIT license.

fps_limit=0
vsync=0
gl_vsync=0
legacy_layout=true
background_alpha=0
alpha=1
font_size=80
background_color=020202
text_color=FFFFFF
position=center
width=5
custom_text=+

offset_x = 945
offset_y = 520
# idk why the above is actually centered, rather than the below
# offset_x = 960
# offset_y = 540
toggle_hud=Shift_R+F12
toggle_logging=Shift_L+F2
upload_log=F5
StellarHarbour commented 3 months ago

I mentioned that we could adopt its approach to enhance software. However, I’m not enthusiastic about repurposing tools designed for specific tasks to serve other purposes, especially when switching between them isn’t straightforward and requires dealing with configurations and other complexities

rpdelaney commented 3 months ago

I agree. That's why we are all here, I think

zkxs commented 3 months ago

Some guys recommended to use MangoHUD as crosshair overlay, and my first thought was that MangoHUD is overlay by itself for sure, can't you use approaches they used?

This looks like the PR where MangoHud added Wayland hotkey support: https://github.com/flightlessmango/MangoHud/pull/1093. Looking at both the PR comments and the diff, I'm not really following what's being done here, but I suspect that MangoHud is hooking into the actual Vulkan/OpenGL rendering, which means it can steal a pointer to the X11/Wayland window and listen to input as a foreground window. Simple Crosshair Overlay doesn't do this: it's just an always-on-top, borderless window that's mostly transparent. The pros of this approach are it's pretty simple and it works on any non-fullscreen-exclusive application, even when there's anticheat preventing you from hooking into the game process. The cons are I can't do things that require me to be hooked into the game process, and not being fullscreen-exclusive incurs a performance cost (but one that many users pay already for the easier Alt+Tabbing). I'm not interested in rewriting Simple Crosshair Overlay to use Vulkan/OpenGL hooking at this time, so any solution will have to be one that works for an always-on-top window overlay.

In https://github.com/ostrosco/device_query/issues/77#issuecomment-1811749945, the device_query dev states

Wayland has a security model that prevents accessing the keyboard and mouse state unless you're the active program so something like device_query won't be able to access global keyboard and mouse state this way. Only with root access and a different method of querying the keyboard and mouse state could we do so in Wayland. Sorry.

I'd imagine Wayland intentionally omits APIs that would allow non-foreground windows to capture keypresses because that's just a keylogger waiting to happen. That said, it appears someone else has gone and made a PoC for the root access method mentioned above: https://github.com/helium18/keylogger/. So I guess the question is this: would Wayland users really be okay with it if Simple Crosshair Overlay required root access so it could read keyboard inputs from the background? Even though this is technically possible and helium18 has provided a PoC, I'm not sure if this is a feature people would be comfortable using... and I'm not sure if it this is a feature I'd be comfortable writing. I don't want to run as root, and I really shouldn't need to.

In my previous comment I mentioned there were three options we could take to work around the Wayland issues:

  1. ditch keyboard input entirely (good luck adjusting the crosshair position in a way that doesn't suck)
  2. move all keyboard input to a foreground-window-only method (but then how do I get the window focused in the first place?)
  3. create completely custom background input capture code just for Wayland, that'd have to run as root. To my knowledge no one has made a library that does this yet.

Option 3 is now looking technically possible, but I don't like it. If anyone can think of a way to keep the software usable with options 1 or 2 I'd love to hear it. My only thought so far is maybe I just let you Alt-Tab to the crosshair overlay window so you can send me foreground keypresses? I'm intentionally preventing this on Windows (because I don't need to be foregrounded on Windows) but I actually can't prevent this on Linux/Max as far as I know.

StellarHarbour commented 3 months ago

What if we split creating preset and using/applying it do different parts so main part won't need Keyboard input?

zkxs commented 3 months ago

The whole idea of the keyboard input is:

  1. I frequently want to toggle the crosshair overlay on and off. Maybe I'm alt tabbing out of the game, maybe some parts of the game don't need the overlay, etc.
  2. Being able to nudge the crosshair around by 1 pixel is SUPER nice. So many games have weird quirks regarding what pixel the bullet actually goes towards, and sometimes it even varies from gun to gun.

These are things I personally do quite a bit when I'm using Simple Crosshair Overlay (god the more I type that name the more I regret how long it is), and I'm reluctant to make the software less usable for myself by removing the hotkey functionality. While I could keep things as-is on Windows and do things totally differently on Linux, that starts to become a maintenance nightmare. When I found the device_query library docs which state it works in "Windows, Linux on X11, and macOS" I was pretty happy, because it meant I could just use one simple API to get keyboard input... or so I thought.

The only option that doesn't completely suck is I keep the keyboard input around, but make it foreground-only. This means I could throw away the concept of "Adjust Mode", because if the window is in the background then it doesn't need to ignore keypress events: it doesn't get them in the first place. The downside is now the user will need to deal with somehow focusing/unfocusing the simple-crosshair-overlay window. This approach would let me drop device_query, solving that particular problem on Linux.

StellarHarbour commented 3 months ago

That sounds like a reasonable approach. We could start and then explore ways to further improve it.

StellarHarbour commented 3 months ago

https://www.gamingonlinux.com/2024/04/mangohud-072-rc1-adds-wayland-keybinds-a-time-label-and-bug-fixes/ Mangohud getting update, maybe it max help you

StellarHarbour commented 2 months ago

Hey, new Fedora and Ubuntu LTS versions coming next week, is there any progress and how we can help?