search

termux / termux-x11

Termux X11 add-on application.
https://termux.dev
GNU General Public License v3.0
2k stars 303 forks source link

Enable Gboard CJK keyboards #620

Closed knyipab closed 3 months ago

knyipab commented 5 months ago

~Relates to #115 and #147~.

Add a preference option "Workaround to enable CJK Gboard".

knyipab commented 5 months ago

Incorrect text UTF8 text sometimes sent to X clients. Not a bug of this commit, instead there are likely some problems in mInjector.sendTextEvent(). Lemme investigate and will reopen after fixing.

knyipab commented 5 months ago

Cannot reproduce the aforementioned bug after re-runtermux-x11 command in Termux. In addition, it's not a bug of this PR, so I re-open this PR.

Only for record: as far as I can tell, there may be bug in XCB receiver side. Log by Java_com_termux_x11_LorieView_sendTextEvent shows that correct character is sent. However, incorrect characters shown in XFCE DE is some characters that was typed before. I tested that termux-x11 session for more than a day and perhaps changes some internal state of XCB receiver side, causing the bug. Anyways, I can't reproduce that at the moment.

knyipab commented 5 months ago

Sorry for confusion. I will stop editing here and look for @twaik your comments on this "Workaround to enable CJK Gboard" option.

More testing are done as below.

Device Android Ver. GBoard Ver. LangTag detection to activate CJK Gboard
Samsung Z Fold5 14 14.1.04.621126403-release-arm64-v8a Working
Samsung S21 14 14.0.11.612796517-release-arm64-v8a Working
Pixel 4a 13 14.0.11.612796517-release-arm64-v8a Not working, blank getLanguageTag()
Pixel 4a 13 14.1.04.621126403-release-arm64-v8a Not working, blank getLanguageTag()
Emulated (AVD Pixel 4) 13 14.1.04.621126403-release-x86_64 Not working, blank getLanguageTag()
Emulated (AVD Pixel 8) 14 12.4.05.482060964-preload-x86_64 Not working, blank getLanguageTag() and requestCursorUpdates() not even called
Emulated (AVD Pixel 8) 14 14.1.04.621126403-release-x86_64 Working

This workaround is by no mean elegant but probably the only existing workaround siince being last discussed in 2021 https://github.com/termux/termux-app/issues/1539. Bit more context of the problem and welcome smarter workaround: the TYPE_TEXT_VARIATION_NORMAL can enable CJK, emoji and clipboard input from Gboard, but at the same time activates Gboard word suggestion causing backspace not working and the word suggestion can be suppressed through restartInput()/invalidateInput(), but does not work well for non-CJK Gboard such as English (when type fast and two keypresses commitText() so close in time, one of them will be ignored). Therefore, the getLanguageTag() detection is a workaround to set TYPE_TEXT_VARIATION_NORMAL only when it is needed (i.e. CJK IME).

Grateful for merging given importance to Gboard CJK users. I don't mind putting it onto an "Experimental" section of perefernces.xml. Thanks.

twaik commented 5 months ago

Relates to #115 and #147.

Not related. That happened because Xwayland (used a long time ago) did not have any instruments to input custom Unicode symbols which are not in current keymap.

Only for record: as far as I can tell, there may be bug in XCB receiver side.

There is no "XCB receiver" in termux-x11. There is some custom light weight protocol to avoid XCB dependency (which adds 1MB+ for every architecture variant and 4.5MB to universal build).

However, incorrect characters shown in XFCE DE is some characters that was typed before.

Termux:X11 sends characters the same way tigervnc-server and x11vnc do. It maps characters to keycodes with no associated keysyms (https://github.com/termux/termux-x11/blob/6fbd33211c8c60be852bb9f0023f735774399611/app/src/main/cpp/lorie/InputXKB.c#L52-L54) and sends key event to application. This method is not really reliable but it works for most text editors and does not interfere with existing hotkey handling implementations.

I did not find better way to implement this, x11vnc way is the best option I found.

knyipab commented 4 months ago

Thanks. Since the bug is really not a problem of this PR and neither Gboard, I opened #625 for test result and further discussion.

Back to this PR, I tested together with #617 (which makes available the 3-finger swipe up gesture to activate soft keyboard in DeX mode) and commit another workaround for DeX mode in which InputConnection#requestCursorUpdates() is not called by Gboard at all.

Any adjustment to this PR you would like me to do before merging?

813ethan commented 3 months ago

any chance this would be merged? working perfectly for me with xfce4 for this branch with traditional chinese gboard

knyipab commented 3 months ago

twaik may have his/her own consideration.

After months of use, apart from the requirement of "Android 14 + Gboard 14" and the input character issue (which is unrelated to Gboard), there are found to be three issues but I haven't had time to look into and solve:

Still, I always think that it would be better to have option than none. I rather to write in the option description to tell user that there is buggy behaviour and may lock your Alt key.

twaik commented 3 months ago

Currently I am working on some other changes, I'll review it again a bit later. Also both this and #617 must be rebased (and the latter one has unresolved reviews). Termux:X11 code has been pretty much changed since prs were opened.

knyipab commented 3 months ago

Rebased.

The Alt issue can be resolved by user going to Gboard settings to disable "Emoji with physical keyboard / Physical Alt to show emoji keyboard".

knyipab commented 3 months ago

Indeed, for those Gboard users who turn off suggestion and autocompletion for their English keyboard (and vaguely other letter-based IMEs), the lang subtype detection (mIMM.getCurrentInputMethodSubtype().getLanguageTag()) and mImeCJK would be unnecessary and LorieView can simply set InputType.TYPE_TEXT_VARIATION_NORMAL, and then users can enjoy voice input and emoji input. Edited: And that won't require Android 14 + Gboard 14.

However, I am not that kind of no-suggestion & no-autocompletion user. I am just saying it and see if you may want to extend an option for those users to force InputType.TYPE_TEXT_VARIATION_NORMAL.

Thanks for your time and consideration.

twaik commented 3 months ago

Pay attention to the review please.

twaik commented 2 months ago

Probably it can be fixed bytusing IBus methods. AFAIK it can let us send unicode symbols without mapping them to keys and avoid this weird behaviour.

knyipab commented 2 months ago

It seems to require ibus to be on the deps of termux-x11-nightly package and start an ibus daemon. Is that right?

Another info: previous test results in #625 show that the implementation tigervnc + realvnc viewer can be 100% reproducible and output is reliable, unlike Termux:X11's unpredictable weird output. (Grateful for letting me know if their approach has sacrificed any features, but) Their approach is almost perfect except that the support to some specific language characters is lacking, which doesn't seem to me to be an implementation limitation. Anything we can possibly learn from their approach?

twaik commented 2 months ago

It is pretty much weird, because I compared termux-x11's implementation with the original one and did not find any differences except function names, dropped upper-lower feature in lorieAddKeysym (because most likely there will be no difference), pressKey replaced by QueueKeyboardEvents, some logging stuff changed, dropped Shift+Alt mapping to META stuff. But generally they are equal.

knyipab commented 2 months ago

No expertise, perhaps about the frontend side (RealVNC) (??).

But I think clues can be found in the xev log files in #625: x11-zh-sc.log and vnc-zh-sc.log. One observable difference is that termux-x11 sends some events between each characters: 

MappingNotify event, serial 51, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 203, count 1

MappingNotify event, serial 51, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 203, count 1

There may be more difference but I haven't probed into.

twaik commented 2 months ago

MappingEvent is event telling programs to associate some keycode (one in this case) to specific unicode symbol.

knyipab commented 2 months ago

Nonetheless, seems that no MappingNotify (but MapNotify) is seen in the log of tigervnc. Where is MappingNotify produced in the termux-x11 code base? I can remove it and test.

twaik commented 2 months ago

Main entry point for unicode input is lorieKeysymKeyboardEvent. But in the case tou remove it keyboard input will be possible only with Prefer scancodes option.