Add proper support for Windows Ink

[Ka1zza]

Short Summary

Not sure if this is a bug or perhaps should be a feature request. I like to use my Win11 2in1 for notes etc in tablet mode. I used to use Fly-Pie on gnome(wayland) which worked great. However it seems that kando is not tracking the pen location. The menu opens at the last location the mouse cursor was at, not the current position of the pen cursor. Only work around at the moment is to centre the menu.

Steps to Reproduce the Issue

1.Trigger menu by hot key (with pen this is mapped to a barrel button and can be triggered while hovering or when the pen tip touches the screen)

2. Menu opens where the mouse cursor last appeared, not under the pen tip.

Kando Version

v1.2.0

Installation Method

Via an installer downloaded from a release on GitHub

Desktop Environment

Windows

Environment Version

Windows 11

Additional Information

No response

[Schneegans]

Hi there! Thanks for the report. Some users have reported issues when the Windows Ink function is enabled. Could you try disabling Windows Ink and see if this changes anything?

[Ka1zza]

Looks like there's no simple way to turn off Windows Ink for my Dell Active Pen. I can disable it in the group policy editor and Kando now works on the desktop. However as soon as I use my pen in an app Kando can't find my pen and the menu appears elsewhere. Looks like it is still using Windows Ink in some apps.

Any chance of being able to support Windows Ink?😅

[Schneegans]

Any chance of being able to support Windows Ink?😅

Well, I guess so, but I do not have the hardware to test this :sweat_smile:

So either someone with the appropriate hardware and coding skills has to show up or somebody has to donate a cheap Windows tablet to me :money_mouth_face:

[Ka1zza]

I have the hardware to test but not the coding skills 😅 I have done some basic coding in python, html, java and was looking at learning Rust. Maybe I'll have a crack at it... haha For now my feature request of custom menu locations #546 would be handy. I usually use my pen on the right side of the screen when note taking so being able to move the menu would be helpful. But I do understand this is might create more problems than its worth.

[Schneegans]

I found this open bug report: https://issues.chromium.org/issues/40935593

So this seems to be an Electron / Chromium issue. On other platforms, Kando does not use GetCursorScreenPoint() either (as it's unreliable) but uses custom native C++ implementations. So I think we will need to add a custom C++ module for this as well...

So if anybody wants to give this a shot, I can provide some pointers.

[Ka1zza]

I would like to give it a go. I've only done the basics with C++ but am willing to try. I did find this which seems promising: https://learn.microsoft.com/en-us/windows/win32/inputmsg/wmpointer-reference I'm not sure how to make use of it yet though. Any pointers would be appreciated. I did join the discord too if that makes things easier.

[Schneegans]

Cool! To implement this, you will have to modify the getActiveWindow() method to also return the latest mouse (or pen) position. This method is called whenever a menu is opened here.

So I would suggest renaming the method to getWMInfo() like in the X11 Linux backend. In that backend, the method is called here. So you could simply do it the same way on Windows.

Whenever you change something in the C++ code, you have to run npm install and npm start again to recompile the native code.

Feel free to ask any follow-up questions in the dev-chat on Discord!

[Ka1zza]

Thanks Schneegans! I'll give it my best shot!