Closed tpstevens closed 8 months ago
Update: it works properly on Debian 12 using XFCE4.
I suspect there's nothing that can be done about this, at least from the Kanata side, with the LLHOOK mechanism.
The SDL2 library probably detects the synthetic inputs and decides to handle them differently than real inputs.
The only workaround would likely be use interception (* see warnings).
Unless of course someone with more SDL2 knowledge can correct me, maybe there are other knobs to turn.
I'm not super familiar with SDL2's inner workings, but I'll see what I can find out. I'm surprised nobody has run into this so far -- SDL is a pretty popular framework for building games and game engines on.
I naively reviewed SDL2's source, and it reads separate Windows events for keydown, keyup, and text input. Where in kanata's source does it generate the event for Windows?
@tpstevens did you solve this problem?
I've been using kanata for 3 months and this has been an issue for me. Usually, I just close kanata which is far from ideal. Interception is also not an option for me, because some games ban it (from what I've researched, many bots use it).
@jtroo I do not know much about LLHOOK or SDL2, but, in my case, the inputs from kanata were recognized, it's just action keys (if I may call that) like enter or space, for example, did not work as intended. I even tested writing some text inside the applications and all keys were normal, including space. I did not test w, a, s, d keys, but I can test it, if it's relevant. What tpstevens said about inputs being read as text makes sense. Do you think is there a way to solve this? Like forcing to seng the input? I saw you linked the 'send_key_sendinput', but my knowledge is limited, I can't do anything with that
One could try playing around with adding the Unicode or Scancode flags: https://learn.microsoft.com/en-us/windows/win32/api/winuser/ns-winuser-keybdinput
And then compile and test.
One could try playing around with adding the Unicode
I've started with your suggestion about unicode, however encountered a problem from the start, which sounds basic, but couldn't find anywhere in code, docs or discussions: how to insert a unicode from the hex value? Is it even possible?
I've tested the common ways for utf-8 and utf-16, but no success. For example, to space character: \u0020, \u0020, u0020, \u{0020}, "\u0020". All of them gave the error "unicode expects exactly one unicode character as an argument"
One could try playing around with adding the Unicode
I've started with your suggestion about unicode, however encountered a problem from the start, which sounds basic, but couldn't find anywhere in code, docs or discussions: how to insert a unicode from the hex value? Is it even possible?
I've tested the common ways for utf-8 and utf-16, but no success. For example, to space character: \u0020, \u0020, u0020, \u{0020}, "\u0020". All of them gave the error "unicode expects exactly one unicode character as an argument"
Based on your comment, my understanding is you're trying the Unicode action in the Kanata config? My comment was more playing with the code using these flags:
Flag | description |
---|---|
KEYEVENTF_SCANCODE 0x0008 | If specified, wScan identifies the key and wVk is ignored. |
KEYEVENTF_UNICODE 0x0004 | If specified, the system synthesizes a VK_PACKET keystroke. The wVk parameter must be zero. This flag can only be combined with the KEYEVENTF_KEYUP flag. For more information, see the Remarks section. |
At this code: https://github.com/jtroo/kanata/blob/b585a5e507d5a2a9326961c6ee06234e730a7ae6/src/oskbd/windows/mod.rs#L62
Based on your comment, my understanding is you're trying the Unicode action in the Kanata config?
I was trying exactly that.
My comment was more playing with the code using these flags:
Ok, I'll give a look into this, thanks
@jtroo your hint was on point. Problem solved.
For future users, change kb_input.wVk = code;
into
let mut special_keys_hash:HashMap<u16, u32> = HashMap::new();
special_keys_hash.insert(0x20, 0x0020); //space key
special_keys_hash.insert(0x0D, 0x000D); //enter key
special_keys_hash.insert(0x1B, 0x001B); //escape key
if special_keys_hash.contains_key(&code) {
let code_u32 = special_keys_hash.get(&code).copied().unwrap();
kb_input.dwFlags |= KEYEVENTF_SCANCODE;
kb_input.wScan = MapVirtualKeyA(code_u32, 0) as u16;
} else {
kb_input.wVk = code;
}
To get the values of each key, look into this site: Virtual key codes Full function:
fn send_key_sendinput(code: u16, is_key_up: bool) {
unsafe {
let mut kb_input: KEYBDINPUT = mem::zeroed();
if is_key_up {
kb_input.dwFlags |= KEYEVENTF_KEYUP;
}
//kb_input.wVk = code;
let mut special_keys_hash:HashMap<u16, u32> = HashMap::new();
special_keys_hash.insert(0x20, 0x0020); //space key
special_keys_hash.insert(0x0D, 0x000D); //enter key
special_keys_hash.insert(0x1B, 0x001B); //escape key
if special_keys_hash.contains_key(&code) {
let code_u32 = special_keys_hash.get(&code).copied().unwrap();
kb_input.dwFlags |= KEYEVENTF_SCANCODE;
kb_input.wScan = MapVirtualKeyA(code_u32, 0) as u16;
} else {
kb_input.wVk = code;
}
let mut inputs: [INPUT; 1] = mem::zeroed();
inputs[0].type_ = INPUT_KEYBOARD;
*inputs[0].u.ki_mut() = kb_input;
SendInput(1, inputs.as_mut_ptr(), mem::size_of::<INPUT>() as _);
}
}
Thanks for doing the investigation work.
It's probably worth investigating at some point in the future if using KEYEVENTF_SCANCODE
for all (or most, if not possible for all) keys would be better than what kanata is doing now.
Will that properly handle holding a key down for a long time and then releasing?
On Tue, Oct 17, 2023 at 3:46 PM, jtroo @.***(mailto:On Tue, Oct 17, 2023 at 3:46 PM, jtroo < wrote:
Thanks for doing the investigation work.
It's probably worth investigating at some point in the future if using KEYEVENTF_SCANCODE for all (or most, if not possible for all) keys would be better than what kanata is doing now.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>
Not sure what the context is around that, I'm not aware of any issue around holding a key for a long time.
The original issue that I discovered was that kanata wasn't sending keydown and keyup events (at least how SDL2 receives them). But I think I misread the sample code above -- it looks like the SendInput function is sent for the up and down event, while I originally thought it was only for keys up. So I'd expect all keys to work properly if KEVENTF_SCANCODE
was applied.
It's probably worth investigating at some point in the future if using KEYEVENTF_SCANCODE for all (or most, if not possible for all) keys would be better than what kanata is doing now.
wScan seems to simulate a key more realistically than wVk, it does not send directly a virtual code like wVk. It sends a code conversion from the virtual code to what the hardware expects. The conversion is made from MapVirtualKeyA, which I don't know exactly how it is done. Not even sure if it varies between hardware. As an addition, I'd guess it works for all keys, since you can convert all of virtual keys.
I have no idea about performance. For sure my test implementation is not the best, I'm even creating a hash on every key press, although it was not enough for me to notice any change in time at all.
What I didn't solve yet is how to use modifier keys (ctrl+c, for example) which are still bugged in my app. However, it's still so specific for my use that I just didn't care to learn how kanata does it and translate from wvk to wscan. From what I saw, it just seems to use an extended version table of virtual key
Will that properly handle holding a key down for a long time and then releasing?
Yes. You can hold down and it'll repeat the key like expected. Below is an example from windows_key_tester of me holding space key
Edit: windows key was bugged, so I had to come wih an if for it. Also changed code
Btw, I've been testing the new method for all keys for the past 3 days and didn't notice any change at all besides everything also working on the app I wanted (keys like enter, space and all modifiers works normal now). Here is the new code for send_key_sendinput function
fn send_key_sendinput(code: u16, is_key_up: bool) {
unsafe {
let mut kb_input: KEYBDINPUT = mem::zeroed();
if is_key_up {
kb_input.dwFlags |= KEYEVENTF_KEYUP;
}
//if for windows left key
if code == 0x5B {
kb_input.wVk = code;
} else {
let code_u32 = code as u32;
kb_input.dwFlags |= KEYEVENTF_SCANCODE;
kb_input.wScan = MapVirtualKeyA(code_u32, 0) as u16;
}
let mut inputs: [INPUT; 1] = mem::zeroed();
inputs[0].type_ = INPUT_KEYBOARD;
*inputs[0].u.ki_mut() = kb_input;
SendInput(1, inputs.as_mut_ptr(), mem::size_of::<INPUT>() as _);
}
}
I have added the SCANCODE
-using code under a feature flag win_sendinput_send_scancodes
for now and plan to build an extra binary with it for some number of releases so that people can experiment with it if desired.
Hey, @jtroo. Just saw your comment, not that active on git. 😓 I've been using the scan method since then and made some tweaks along the way. The last change was in december and everything seems fine. The win key, for example, is working.
Here are the comments I left to my future self so I could remember what I was doing: "All the keys that are extended are on font 1, inside the table on column 'Scan 1 Make' and start with '0xE0'. To obtain the scancode, one could just print 'kb_input.wScan' from the function below. Font 1: https://learn.microsoft.com/en-us/windows/win32/inputdev/about-keyboard-input#scan-codes To obtain a virtual key code, one could just print 'code' from the function below for a key or see font 2 Font 2: https://learn.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes
For example, left arrow and 4 from numpad. For the scancode, they have the same low byte, but not the same high byte, which is 0xE0. LeftArrow = 0xE04B, keypad 4 = 0x004B. For the virtual code, left arrow is 0x25 and 4 from numpad is 0x64. There is a windows function called 'MapVirtualKeyA' that can be used to convert a virtual key code to a scancode."
What I imagine is that SDL2-based apps use the scancode, not sure why they use the wrong one though (I even read some problems very similar to ours with people programming games, because the inputs were being translated wrong). With my last code posted, I converted the virtualcode to scancode, but passed only the low byte to 'kb_input.wScan' and the program assumed 0x00 as the high byte for all the keys, resulting in all the extended keys with the wrong output.
I've made an array with the virtualcodes to be able to identify the extended keys and change the high byte. The array was made from looking into font 1 to know whick keys were extended and got the virtualcode inside font 2.
If there is any question, don't mind asking. Here is the new code:
fn send_key_sendinput(code: u16, is_key_up: bool) {
const EXTENDED_KEYS: [u16; 35] = [
0xb1, 0xb0, 0xa3, 0xad, 0x8c, 0xb3, 0xb2, 0xae, 0xaf, 0xac, 0x6f,
0x2c, 0xa5, 0x24, 0x26, 0x21, 0x25, 0x27, 0x23, 0x28, 0x22, 0x2d,
0x2e, 0x5b, 0x5c, 0x5d, 0x5f, 0xaa, 0xa8, 0xa9, 0xa7, 0xa6, 0xac,
0xb4, 0x13,
];
unsafe {
let mut kb_input: KEYBDINPUT = mem::zeroed();
if is_key_up {
kb_input.dwFlags |= KEYEVENTF_KEYUP;
}
let code_u32 = code as u32;
kb_input.dwFlags |= KEYEVENTF_SCANCODE;
kb_input.wScan = MapVirtualKeyA(code_u32, 0) as u16;
let is_extended_key: bool = EXTENDED_KEYS.contains(&code);
if is_extended_key {
kb_input.wScan = (0xE0 << 8) | kb_input.wScan;
kb_input.dwFlags |= KEYEVENTF_EXTENDEDKEY;
}
let mut inputs: [INPUT; 1] = mem::zeroed();
inputs[0].type_ = INPUT_KEYBOARD;
*inputs[0].u.ki_mut() = kb_input;
SendInput(1, inputs.as_mut_ptr(), mem::size_of::<INPUT>() as _);
}
}
Thanks for the updated code!
Not work in Terraria, any key in layer can not accept by game. I can only close Katana to solve it. So sad...
How about keep a black name list, in which kanata not active in those program to solve it?
Not work in Terraria, any key in layer can not accept by game. I can only close Katana to solve it. So sad...
Which variant of kanata did you try? Please try one of: winiov2 or scancode from the release page, if you have not tried it.
Not work in Terraria, any key in layer can not accept by game. I can only close Katana to solve it. So sad...
Which variant of kanata did you try? Please try one of: winiov2 or scancode from the release page, if you have not tried it.
I tried the winiov2 edition, it works well, thanks for you work!
how to compile the winiov2 version myself?
same question for the scancode version, maybe --features win_sendinput_send_scancodes, win_llhook_read_scancodes
?
Requirements
Describe the bug
On Windows 10, kanata sends events that get treated by SDL (and probably the OS?) as text input events but not keyup and keydown events.
Without kanata running:
With kanata running:
I've tested with my own games built with SDL2 (which is how I generated the above logs) and other games that use SDL2. I'm happy to test on Debian as well.
Relevant kanata config
No response
To Reproduce
Expected behavior
kanata should send keydown and keyup events. I'd expect that the keydown might come with a little delay after kanata has decided what the key should resolve to.
Kanata version
1.4.0
Debug logs
No response
Operating system
Windows 10
Additional context
No response