Kitty slows down and eventually become unresponsive when fcitx5 is running

Describe the bug When fcitx5 is running, kitty slows down, and when run with --debug-keyboard, it spams the following in the log

[0.127] Loading new XKB keymaps
[0.131] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2
[0.131] Loading new XKB keymaps
[0.135] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2
[0.135] Loading new XKB keymaps
[0.140] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2
[0.140] Loading new XKB keymaps
[0.145] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2
[0.145] Loading new XKB keymaps
[0.150] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2
[0.150] Loading new XKB keymaps
[0.154] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2
[0.154] Loading new XKB keymaps
[0.159] Modifier indices alt: 0x3 super: 0x6 hyper: 0xffffffff meta: 0xffffffff numlock: 0x4 shift: 0x0 capslock: 0x1 control: 0x2

To Reproduce Steps to reproduce the behavior:

Install fcitx5 and kitty, I'm compiling kitty from source (commit 8183e2f538ea9231f75cab345e0d1c0c9d35343a), but it's still reproducible on 0.32.2, 0.33.1, and 0.34.1.
Run fcitx5-remote -t to start up fcitx5
Run kitty --debug-keyboard --config /dev/null 2>&1 | tee kitty.log, and optionally move your focus around
See log spamming, as shown above.
After a long time, kitty starts slowing down, and input becomes unresponsive, and uses up a lot of CPU.

Screenshots If applicable, add screenshots to help explain your problem.

Environment details

kitty 0.34.1 created by Kovid Goyal
Linux ulysses 6.8.0-pf7-ryan #1 SMP PREEMPT_DYNAMIC Wed May  1 16:47:27 CST 2024 x86_64

This is ulysses (Linux x86_64 6.8.0-pf7-ryan) 20:44:08

DISTRIB_ID="Gentoo"
Running under: Wayland (sway version 1.9) missing: blur
Frozen: False
Paths:
  kitty: /usr/bin/kitty
  base dir: /usr/lib64/kitty
  extensions dir: /usr/lib64/kitty/kitty
  system shell: /bin/zsh
Loaded config files:
  /dev/null

Config options different from defaults:

Important environment variables seen by the kitty process:
    PATH                                /usr/lib/ccache/bin:/home/ryan/.local/share/zinit/plugins/direnv---direnv:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/cuda/bin:/home/ryan/.local/bin:/home/ryan/.scripts:/home/ryan/.cache/go/bin:/home/ryan/.local/share/nvim/mason/bin:/home/ryan/.local/share/zinit/plugins/zpm-zsh---colors/bin
    LANG                                en_US.utf8
    EDITOR                              /bin/nvim
    SHELL                               /bin/zsh
    GLFW_IM_MODULE                      fcitx 
    DISPLAY                             :0
    WAYLAND_DISPLAY                     wayland-1
    USER                                ryan
    XCURSOR_SIZE                        24
    XDG_SEAT                            seat0
    XDG_SESSION_TYPE                    wayland
    XDG_SESSION_CLASS                   user
    XDG_SESSION_ID                      3
    XDG_RUNTIME_DIR                     /run/user/1000
    XDG_VTNR                            7
    XDG_CONFIG_DIRS                     /etc/xdg
    XDG_DATA_DIRS                       /home/ryan/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
    XDG_CACHE_HOME                      /home/ryan/.cache
    XDG_CONFIG_HOME                     /home/ryan/.config

Additional context Some relevant installed packages:

[I] x11-libs/libX11 (1.8.9@04/29/2024): X.Org X11 library
[I] gui-wm/sway (1.9@04/29/2024): i3-compatible Wayland window manager
[I] app-i18n/fcitx (5.1.9(5)@04/26/2024): Fcitx 5 is a generic input method framework
[I] gui-libs/wlroots (0.17.2(0/17)@04/14/2024): Pluggable, composable, unopinionated modules for building a Wayland compositor

When fcitx5 is not running, kitty's log won't lag or show such spam, which is why I think the log is related to the lag. However, I don't know what's the exact reason for the slow down.

kovidgoyal / kitty

Kitty slows down and eventually become unresponsive when fcitx5 is running #7412