search

keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
372 stars 102 forks source link

bug(linux): Chrome(ium) / Electron apps don't "delete surrounding text" #5510

Closed rik-shaw closed 1 year ago

rik-shaw commented 2 years ago

Describe the bug In Ubuntu / Wasta-Linux (all versions), when using Keyman keyboards or older ibus-kmfl keyboards in Chrome(ium) - basically ALL browsers except Firefox at this point, and also Electron apps (such as vscode or skype, or the upcoming PTLite), then multiple key sequences that need "delete surrounding text" do not delete the initial character when producing the resulting character.

For example, we use ; as a "visible trigger" for users, so ; n is typed to produce ŋ. When typing the sequence, the ; should display, then when n is pressed the ; should be deleted and ŋ is inserted. However, ;ŋ is displayed.

For another example, if you use the "SIL Ethiopic" keyboard, typing m u should first display ም when m is pressed, then delete it and replace with ሙ when u is pressed. Instead, the ም isn't deleted and ምሙ displays.

I believe this may be discussed in other issues, but I wasn't certain which one to add to, so this may be listed as a duplicate.

Keyman for Windows/macOS/Linux:

Additional Note: In Firefox, when using "Google Docs" the same problem exists. I have NOT seen it in any other site, so I think it is specific to Google. It may be best for me to report that as a separate issue.

rik-shaw commented 2 years ago

I also submitted a bug to ibus itself: https://github.com/ibus/ibus/issues/2354 and will report back if there is any feedback from them. Until now it hasn't been too critical but with Paratext Lite (Linux) being Electron (Chromium) based it is critical we be able to keyboard in it.

rik-shaw commented 2 years ago

@mcdurdin and @darcywong00 I tagged you on the ibus issue (link above). In short, they are suggesting I don't rely on 'delete surrounding text' but rather keep the ; in "pre-edit". This may simplify the headache of Linux keyboarding if we don't need "delete surrounding text". However, when it then comes to Ethiopic keyboards where a single press of m should produce ም but m a should remove ም and replace with ማ would this "preedit" approach still work?

Basically I am not sure of what preedit is and if Keyman can use it, possibly having me modify keyboards to use preedit rather than delete surrounding text. Your advice and recommendation are much appreciated! :-) Thanks again for all your work.

mcdurdin commented 2 years ago

As I noted in https://github.com/ibus/ibus/issues/2354#issuecomment-925383900, I don't think preedit is really viable for Keyman.

ermshiperete commented 2 years ago

See also Chromium issue: https://bugs.chromium.org/p/chromium/issues/detail?id=1123174

rik-shaw commented 2 years ago

Not entirely related, but wondering if there are plans for Keyman support in ChromeOS? I don't know how ChromeOS handles keyboard input methods?

Also not sure if newer ChromeOS devices supporting Linux and Android app installation means that Keyman could be used that way? (On this second point, I suspect not as I think the Linux / Android app support is via containers... lxc or other not certain but regardless, containerized approach which may then not be available outside the container 'sandbox')

mcdurdin commented 2 years ago

@rik-shaw actually completely unrelated 😆 -- it's on our agenda but we don't have any capacity to do any work. In my testing, I found Keyman for Android does work, sorta, on modern ChromeOS, but the code will need some work for complete support, and I've no idea if it would work with Linux apps -- never tried. But in any case, not related to this issue. (Feel free to open an issue on this if it is important going forward!)

rik-shaw commented 2 years ago

@mcdurdin I was mainly wondering if any ChromeOS keyboarding success could help understand more of Linux keyboarding in Chrome, as ChromeOS is basically Chrome running on top of Linux. But in our context ChromeOS in itself is not worth pursuing at this time.

mcdurdin commented 2 years ago

@rik-shaw as far as I can tell, it's a completely different keyboarding stack (perhaps at kernel level it's the same, but there are a whole lotta layers on top of that).

ermshiperete commented 2 years ago

related issue #1489

ermshiperete commented 1 year ago

This is only a problem with ibus > 1.5.17, i.e. since Ubuntu 20.04 Focal.

rik-shaw commented 1 year ago

@ermshiperete could you clarify what versions of ibus / keyman would be needed to get it working in Ubuntu / Wasta-Linux 20.04? Maybe the ibus fix hasn't been released yet so the version isn't known, but once it is, I could work on trying to backport it to 20.04.

ermshiperete commented 1 year ago

@rik-shaw The official ibus version isn't known yet - one of the necessary patches is merged in the upstream version, but no new ibus version released yet, and the other patches are still waiting to be accepted (the upstream maintainer is currently busy with other things, so it'll take a bit longer).

However, I built patched ibus packages that are currently available on llso, pso, and our ppa. You'll recognize the correct ibus version by the postfixed *sil2.1* package version number.

rik-shaw commented 1 year ago

However, I built patched ibus packages that are currently available on llso, pso, and our ppa. You'll recognize the correct ibus version by the postfixed *sil2.1* package version number.

@ermshiperete thank you very much for building the patched versions! I have 1.5.22-2ubuntu2.1sil2.1~focal now installed (in 20.04 of course) and in Firefox, Skype (electron app - flatpak), telegram (electron app - both snap and flatpak versions) it is working well (first character is correctly replaced by the resulting character after 2nd key is pressed)....

.... however in Paratext Lite (electron - snap), Gnome-Terminal (deb), Chromium (flatpak), and Google Docs in Firefox (deb - again rest of firefox like lthis github issue work great with Keyman btw :-) ), the initial characters are not removed (similar to before). Is there an env variable I can set when launching these apps or something like that? The problem exists with keyman and ibus-kmfl keyboards still.

Or maybe additional patches are still needed for compatibility with these apps? Sorry I am a bit confused so it is likely you have explained before but I have misunderstood. Thanks again SO much for everything so far, I am sure we are nearly there.

UPDATE: I'll test in 22.04 and report back if things are not working there as well, since from other bug reports it seems it should WORK there, so maybe this above issue is only in 20.04.

ermshiperete commented 1 year ago

@rik-shaw Unfortunately all snap and flatpak apps won't work since they use their own version of ibus or something - I'm not exactly sure how that works. There are three different parts involved: of course the Keyman engine (ibus-engine-keyman), then the ibus-daemon, and GTK version specific im-ibus.so libraries that get loaded into the client app. For snap/flatpak apps ibus-engine-keyman and ibus-daemon run on the host. For communication with flatpak there's a portal that allows the communication between the container and the host. I guess snap must use something similar. And I assume im-ibus.so is contained in the container and would need to be updated there (which will happen sometime after it became available in the official repos). However, I haven't verified my assumptions, so my current understanding might be wrong.

With gnome-terminal: try and copy/paste the output of gnome-terminal into gedit or LO Writer and see if it's correct there. gnome-terminal has font issues... Also, please try the following: after you opened the terminal, press Enter and only then start typing with Keyman - there have been some oddities with the first character after opening the terminal (please feel free to create a separate issue if you see that behavior since I didn't create one :smile: ).

I haven't tried in Google Docs, so I'm not sure if it'll work there (although I'd think it should).

And yes, please test in 22.04 and create separate issues if things don't work there.

rik-shaw commented 1 year ago

@ermshiperete thanks for the detailed reply above. I just did a dist-upgrade on my main 20.04 system to re-test different electron apps. I did see that newer versions of keyman and ibus-keyman came in (now at 15.0.271-1+focal1), that may be contributing to good things.

For testing I have Skype (deb and snap), Chromium (deb and snap), and VS Code (deb and snap) to compare / contrast how deb and snap work with the updated keyman versions.

App snap keyman snap ibus-kmfl deb keyman deb ibus-kmfl
Chromium doesn't work doesn't work works doesn't work if directly select ibus-kmfl keyboard. BUT if activate a keyman keyboard, then change to an ibus-kmfl keyboard it works??!!??
VS Code works doesn't work if directly select ibus-kmfl keyboard. BUT if activate a keyman keyboard, then change to an ibus-kmfl keyboard it works??!!?? works doesn't work if directly select ibus-kmfl keyboard. BUT if activate a keyman keyboard, then change to an ibus-kmfl keyboard it works??!!??
Skype works doesn't work if directly select ibus-kmfl keyboard. BUT if activate a keyman keyboard, then change to an ibus-kmfl keyboard it works??!!?? works doesn't work if directly select ibus-kmfl keyboard. BUT if activate a keyman keyboard, then change to an ibus-kmfl keyboard it works??!!??
Paratext Lite doesn't work doesn't work n/a n/a

So there are 2 main issues:

  1. Keyman works in some electron apps, not in others: In VS Code and Skype snap versions, keyman works! For VS Code, it is a --classic snap which means it is unconfined. I think this is why it may work. But Skype is NOT confined, is an electron app, and keyman keyboarding does correctly 'delete surrounding text'. Totally confused :-) I will post this feedback to Paratext Lite: maybe they will be able to sort out why it works for Skype but not for PT Lite / Chromium.

  2. At least for me there is still an issue with ibus-kmfl keyboards, which may not be able to be solved since it isn't actively developed / supported anymore. I am willing to forgo using them if only some other parts of keyman weren't so frustrating:

    1. App icon doesn't show in ibus system tray, instead only generic keyman icon
      • issue reported: #7547
    2. Keyboard name in ibus list starts with the first language supported by the keyboard, but our keyboards are country-wide, and showing the first matching language is misleading and even leads to some conflict in some cases. For example, our SIL Ethiopic keyman keyboard shows as "Amharic - SIL Ethiopic" and our Ethiopian Latin keyboard shows as "Hammer-Banna - SIL EL - Ethiopian Latin"
      • issue reported: #7548
    3. Can't install "system wide" to be easily pre-installed as part of our ISO. instead only user-level installs supported.
      • issue reported: #7549
        • note: km-package-install --shared may address my needs, I will need to do some testing.
        • update: yes, km-package-install --shared seems to address this need. With the first two items above addressed it would allow full migration from ibus-kmfl to Keyman. Thanks for the great work over the years!
mcdurdin commented 1 year ago

I am not familiar with how snap packages are built, but for PT Lite, I would hope that they could build their snap with the sil2.1 version of ibus, and it should then work? For Chromium, I am guessing we have to wait for ibus maintainer to accept and merge our patches, and then for those to slowly percolate their way upstream to distros, and to the chromium snap builds. I don't know how long that will take. Seems painfully slow!

2. if only some other parts of keyman weren't so frustrating

Can we track these things you are finding frustrating as new, separate issues (link to them here)? Particularly highlight how kmfl was working better than Keyman for each issue, if that is possible. Details and screenshots really help here. We are preparing our 17.0 planning right now, so timing is perfect. (Note that we have a significant work item around wayland support for 17.0 -- wayland APIs still seem fairly immature and unstable but we have made some progress in support.)

mcdurdin commented 1 year ago

(Note: @ermshiperete is on vacation currently and won't be around for a couple of weeks)

rik-shaw commented 1 year ago

Can we track these things you are finding frustrating as new, separate issues (link to them here)? Particularly highlight how kmfl was working better than Keyman for each issue, if that is possible. Details and screenshots really help here. We are preparing our 17.0 planning right now, so timing is perfect. (Note that we have a significant work item around wayland support for 17.0 -- wayland APIs still seem fairly immature and unstable but we have made some progress in support.)

Sorry I haven't reported these issues before even though being frustrated by them for multiple years. Please forgive me, the last few years haven't been kind to me being able to keep up on better reporting / documenting these issues. I have created bug reports and linked them in the above comment.

mcdurdin commented 1 year ago

@rik-shaw, no worries mate :grin: Reviewing the issues now, they are really clearly documented, thank you!

If you don't mind me asking, did you find the new issue templates to be clear? We've just implemented them using GitHub's new (beta) issue template form functionality.

rik-shaw commented 1 year ago

@mcdurdin yes I liked the new issue template, good work! I sort of stuck to the "Describe the bug" section and left the "steps to reproduce" and "expected behavior" blank, that may have been confusing for you, I may need encouragement to follow directions more closely in the future :-)

About this issue, thanks to the Paratext Lite developer there is a workaround that enables correct deleting of the "character building letter". In short, by launching ParatextLite this way:

GTK_IM_MODULE=xim paratextlite

This also works in the Chromium snap.

The explanation from the developer is that each snap has a core snap it targets (eg. core18 - for 18.04, core20 - for 20.04, etc.), and paratextlite uses the gnome-3-38 snap which targets core20. As noted previously, Ubuntu 18.04 (and thus the core18 snap) doesn't have the Chromium keyboarding issue. The Skype snap targets core18, thus ibus keyboarding appears to work in it (but there have been other improvements to ibus since the 18.04 version so it isn't all perfect in the core18 snap but it doesn't have the regression of not deleting "character building letters").

The developer tested by applying the updated ibus-gtk3 from the Keyman PPA to the gnome-3-38 snap and with that done, Paratext Lite keyboarding worked correctly. So this is the best solution but how to do that correctly is challenging.

rik-shaw commented 1 year ago

Paratext Lite was updated to incorporate the updated ibus-gtk3 version, meaning that "intermediate character building letters" are deleted and replaced with the final desired character. Note as with other Chromium-based apps above deleting of "intermediate character building characters" doesn't work if the user directly selects an ibus-kmfl keyboard, BUT if they instead activate a Keyman keyboard and then change to an ibus-kmfl keyboard it works. Summary: Keyman is the way forward for the best user experience :-)

Thanks again to @ermshiperete @mcdurdin and the PT Lite developer for working through this complicated issue. When the above referenced "minor frustrations" with Keyman on Linux are solved (#7547 and #7548) it will give a "fully functional and fully polished experience" to the user, but at least in the current state with the fix to PT Lite users can now keyboard correctly which is a huge step forward. Thank you again!

mcdurdin commented 1 year ago

That's great news on PT Lite @rik-shaw!

Note that the changes to ibus are two part: #7072 describes a regression that completely broke input in Chromium, visible in Ubuntu 20.04. After some discussion and testing, the ibus developer rolled this change back, although at this point this has not yet made it into a release.

However, the more significant patch is still awaiting review. This patch makes it possible for Keyman to ensure that text output in all applications has guaranteed consistency; in 18.04, it was just 'likely' to be correct, but high system load or other factors could result in incorrect output order. Applications that support 'surrounding text', such as Firefox, were not impacted by this issue. Chromium does not yet support 'surrounding text'. The version of ibus that we are distributing includes this patch.