[Bug] Dead keys don't work properly with mod taps and layer taps under Windows with Wincompose

[jweickm]

I am using homerow mods and other variants of mod taps/layer taps in my German keymap. The German keymap includes three dead keys (keys that change the next key pressed): ^, `, and ´. So when pressing ` (DE_GRV) + E (KC_E) + Space (KC_SPC) repeatedly, I receive the following output:

è è è è è è è è è è 

However, when E is a mod tap like RSFT_T(KC_E), the generated output looks like this:

e ``e è è è e ``e e ``e e 

Tested on two different keyboards

System Information

Keyboard 1: OLKB Planck Rev/6 Keyboard 2: Checkerboards Quark Operating system: Windows 11

Any keyboard related software installed?

• [x] AutoHotKey (Windows)
• [ ] Karabiner (macOS)
• [ ] Other:

Additional Context

Pressing a "normal" E key (KC_E) without any mod taps does not cause this error and produces the normal expected result. I also tried pressing the dead key on my QMK keyboard and pressing the E key on the keyboard of my laptop, which also resulted in the desired output. This led me to believe that there must be some incompatibility between mod taps and (native) dead keys. Any ideas or help is greatly appreciated!

[jweickm]

Additional comment: I was playing around a bit and it turns out that tap dance also interfers with dead keys. So pressing a German dead key and then pressing a tap dance key does not always add the accent on the next character (as would be expected behaviour). As I wrote in the initial post, pressing the dead key and then the alpha key on the computer keyboard works, which means that the dead key is properly registered by the OS. What causes it to "forget" about the dead key when pressing mod taps, I honestly have no clue. I tested this again on a Samsung Android device, here it works properly. So it seems to be some issue in the communication between Windows and QMK keyboard. I also tried it in a vim buffer in WSL. Here, the issue also did not occur.

[jweickm]

I added another comment. I tried disabling Wincompose and this appears to disable the issue. This seems very odd to me, as Wincompose is triggered with the AltGr key on my computer, which is not part of any of the above sequences. Furthermore, it only seems to affect the input from my QMK powered keyboard and not the laptop keyboard. I will also post this issue on the Wincompose Github page.