fix(linux): Fix reordering of output

[ermshiperete]

Quick link to the test results

This change implements a commit queue which allows to control the order of the output. This ensures that any backspace we generate will be processed before the character we're adding.

Requires changes in ibus (surrounding text fix (see #7072) and prefilter change).

• Fixes #1489.
• Fixes #4028.
• Fixes #4029.
• Fixes #4030.
• Fixes #4505.
• Fixes #5510.
• Fixes #6639.

User Testing

Preparations

• The tests should be run on these Linux platforms:

  • GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
  • GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
  • GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
  • GROUP_WASTA: Wasta 20.04 with Cinnamon

• Add the Keyman Test ppa:

  sudo add-apt-repository ppa:keymanapp/keyman-test

• Install all updates

  sudo apt update
  sudo apt upgrade

• Verify that you have the updated ibus version required to test this PR

  • Run the following command

    dpkg -l ibus

  • This shows a table similar to this:

    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name           Version            Architecture Description
    +++-==============-==================-============-===========================>
    ii  ibus           1.5.26-4sil1~jammy amd64        Intelligent Input Bus - core

    The exact version depends on the platform. The important part is the sil in the version number - that needs to be there, otherwise you don't have updated ibus version.

• On Ubuntu 22.04 Jammy, install the Firefox .deb package (not snap) - this is not needed for older distributions

  • Run the following commands:

    sudo apt remove firefox
    snap remove firefox
    sudo add-apt-repository ppa:mozillateam/ppa
    echo '
    Package: *
    Pin: release o=LP-PPA-mozillateam
    Pin-Priority: 1001
    ' | sudo tee /etc/apt/preferences.d/mozilla-firefox
    echo 'Unattended-Upgrade::Allowed-Origins:: "LP-PPA-mozillateam:${distro_codename}";' | sudo tee /etc/apt/apt.conf.d/51unattended-upgrades-firefox
    sudo apt install firefox

  • when testing in Firefox, test in the address bar.

• On Ubuntu 22.04 Jammy, Ubuntu 20.04 Focal and Wasta install the Chromium .deb package (not snap) - this is not needed for Bionic

  • Run the following commands:

    sudo add-apt-repository ppa:saiarcot895/chromium-beta
    sudo apt install chromium-browser

  • when testing in Chromium, open https://keyman.com/keyboards and type in the "Keyboard search" field

• Install Anki

  • Download Qt5 version of Anki (For Bionic use the "Download Anki for 2016+ Linux" version)
  • Run the following commands

    sudo apt update
    sudo apt install zstd
    tar xaf Downloads/anki-2.*-linux-qt5.tar.zst
    cd anki-2.*-linux
    sudo ./install.sh
    sudo apt install libxcb-xinerama0

    (for Bionic, the first line has to be: tar xfx anki-2.*-linux.tar.bz2)

  • When testing in Anki, select e.g. "Tools"/"Study Deck"

• Install build artifacts of this PR

• Reboot

• Install the following keyboards in Keyman:

  • IPA (SIL)
  • Korean KORDA Jamo (SIL)
  • Khmer Angkor
  • Vedic Sanskrit Devanagari Phonetic (ITRANS)

SUITE_WRITER: LibreOffice Writer

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Open LibreOffice Writer.

Tests

• TEST_WRITER_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_WRITER_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_WRITER_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री". (If the result looks wrong, select all text and change the font to "Siddhanta")
• TEST_WRITER_KM: Switch to "Khmer Angkor" keyboard. Type xEjmr. Verify that the output is "ខ្មែរ".
• TEST_WRITER_KM_BS: With the output "ខ្មែរ" still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

SUITE_GEDIT: gedit

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Open gedit.

Tests

• TEST_GEDIT_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_GEDIT_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_GEDIT_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री".
• TEST_GEDIT_KM: Switch to "Khmer Angkor" keyboard. Type xEjmr. Verify that the output is "ខ្មែរ".
• TEST_GEDIT_KM_BS: With the output "ខ្មែរ" still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

SUITE_FIREFOX: Firefox

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Open https://keyman.com/keyboards in Firefox.

NOTE: If the output looks wrong, copy the text from the browser and paste it into gedit and verify that it there.

Tests

• TEST_FIREFOX_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_FIREFOX_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_FIREFOX_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री".
• TEST_FIREFOX_KM: Switch to "Khmer Angkor" keyboard. Type xEjmr. Verify that the output is "ខ្មែរ".
• TEST_FIREFOX_KM_BS: With the output "ខ្មែរ" still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

SUITE_CHROMIUM: Chromium

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Open Chromium browser.

NOTE: If the output looks wrong, copy the text from the browser and paste it into gedit and verify that it there.

Tests

• TEST_CHROMIUM_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_CHROMIUM_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_CHROMIUM_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री".
• TEST_CHROMIUM_KM: Switch to "Khmer Angkor" keyboard. Type xEjmr. Verify that the output is "ខ្មែរ".
• TEST_CHROMIUM_KM_BS: With the output "ខ្មែរ" still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

SUITE_TERMINAL: gnome-terminal

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Open Terminal.

Tests

• TEST_TERMINAL_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_TERMINAL_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_TERMINAL_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री".
• TEST_TERMINAL_KM: Switch to "Khmer Angkor" keyboard. Type xjmEr (Note the different character order!). Copy the output characters from terminal and paste them in gedit. Verify that the pasted text is "ខ្មែរ". (Terminal has a rendering problem with some Khmer characters, so the output in terminal looks wrong, even though Keyman added the correct characters.)
• TEST_TERMINAL_KM_BS: With the output of the previous test still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

SUITE_ANKI: Anki

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Open Anki.

Tests

• TEST_ANKI_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_ANKI_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_ANKI_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री" (If the result looks wrong, copy/paste it in text editor and verify there).
• TEST_ANKI_KM: Switch to "Khmer Angkor" keyboard. Type xEjmr. Verify that the output is "ខ្មែរ".
• TEST_ANKI_KM_BS: With the output "ខ្មែរ" still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

SUITE_SEARCHBAR: Searchbar in gnome-shell

• GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11
• GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11
• GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11
• GROUP_WASTA: Wasta 20.04 with Cinnamon

Press Windows key to open searchbar.

Tests

• TEST_SEARCHBAR_IPA: Switch to "IPA (SIL)" keyboard. Type n>. Verify that the result is "ŋ".
• TEST_SEARCHBAR_KO: Switch to "Korean KORDA Jamo (SIL)" keyboard. Type han<space>geul<space>. Verify that the result is "한글".
• TEST_SEARCHBAR_HI: Switch to "Vedic Sanskrit Devanagari Phonetic (ITRANS)" keyboard. Type shrI. Verify that the result is "श्री".
• TEST_SEARCHBAR_KM: Switch to "Khmer Angkor" keyboard. Type xEjmr. Verify that the output is "ខ្មែរ".
• TEST_SEARCHBAR_KM_BS: With the output "ខ្មែរ" still being displayed, press Backspace key 5 times. Verify that after each keystroke a part of the string is removed and after the 5th backspace the entire word is deleted.

[bharanidharanj]

SUITE_CHROMIUM: Chromium

GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11

• TEST_CHROMIUM_HI (PASSED): Retested this as per Eberhard's suggestion and here is my observation. After pasting the Hindi font into gedit, the font changes and showing the correct output.

  

[bharanidharanj]

SUITE_WRITER: LibreOffice Writer

GROUP_BIONIC: Ubuntu 18.04 Bionic with Gnome Shell and X11

• TEST_WRITER_KM (PASSED): Retested this as per Eberhard's instructions and after changing the font, I was able to see the expected output.

  

[bharanidharanj]

SUITE_FIREFOX: Firefox

GROUP_WASTA: Wasta 20.04 with Cinnamon

• TEST_FIREFOX_IPA (PASSED): Retested this with Keyman 16.0.74-alpha build as per Ebherhard's suggestion and it is working as expected.

  

• TEST_FIREFOX_KO (PASSED): Retested this and I noticed that after pasting it on the gedit, it is showing the correct output.

  

• TEST_FIREFOX_HI (PASSED): Retested this and I noticed that after pasting it on the gedit, it is showing the correct output.

  

• TEST_FIREFOX_KM (PASSED): Retested this in the Keyman Search bar and it is working as expected.

  

• TEST_FIREFOX_KM_BS (PASSED): After Pressing Backspace key 5 times, the entire word is deleted.

[ermshiperete]

@keymanapp-test-bot retest SUITE_CHROMIUM GROUP_FOCAL TEST_CHROMIUM_KO TEST_CHROMIUM_HI GROUP_JAMMY_X11 TEST_CHROMIUM_IPA TEST_CHROMIUM_KO TEST_CHROMIUM_HI TEST_CHROMIUM_KM TEST_CHROMIUM_KM_BS GROUP_WASTA TEST_CHROMIUM_IPA TEST_CHROMIUM_KO TEST_CHROMIUM_HI TEST_CHROMIUM_KM TEST_CHROMIUM_KM_BS SUITE_ANKI GROUP_WASTA TEST_ANKI_HI

[ermshiperete]

@bharanidharanj Please retest TEST_TERMINAL_IPA in Jammy with a slight variation: after you opened gnome-terminal, press the Enter key first, then start with the test (cf #7428).

@keymanapp-test-bot retest SUITE_TERMINAL GROUP_JAMMY_X11 TEST_TERMINAL_IPA

[ermshiperete]

The following failing tests in Jammy are caused by gnome-terminal not supporting surrounding text and deleting clusters instead of single code points (see #7130), so we skip these tests.

SUITE_TERMINAL GROUP_JAMMY_X11

• TEST_TERMINAL_KO skipped
• TEST_TERMINAL_HI skipped
• TEST_TERMINAL_KM skipped
• TEST_TERMINAL_KM_BS skipped

[bharanidharanj]

SUITE_TERMINAL: gnome-terminal

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11

• TEST_TERMINAL_IPA (PASSED): Tested this as per Eberhard's instructions in Jammy (X11) Linux and it is showing the correct output.

  

[bharanidharanj]

SUITE_FIREFOX: Firefox

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11

• TEST_FIREFOX_IPA (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_FIREFOX_KO (PASSED):Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_FIREFOX_HI (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_FIREFOX_KM (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_FIREFOX_KM_BS (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11, after Pressing Backspace key 5 times, the entire word is deleted.

[bharanidharanj]

SUITE_CHROMIUM: Chromium

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11

• TEST_CHROMIUM_IPA (PASSED): Retested this as per the instructions in (Keyman 16.0.76-alpha) Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_CHROMIUM_KO (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_CHROMIUM_HI (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_CHROMIUM_KM (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11 and it is showing the correct output.

  

• TEST_CHROMIUM_KM_BS (PASSED): Retested this as per the instructions in Ubuntu 22.04 Jammy X11, after Pressing Backspace key 5 times, the entire word is deleted.

[bharanidharanj]

SUITE_CHROMIUM: Chromium

GROUP_WASTA: Wasta 20.04 with Cinnamon

• TEST_CHROMIUM_IPA (PASSED): Retested this as per the instructions in (Keyman 16.0.76-alpha) Wasta 20.04 Linux and it is showing the correct output.

  

• TEST_CHROMIUM_KO (PASSED): Retested this in Wasta Linux and it seems that after pasting the fonts in the text editor it shows the correct the output.

  

• TEST_CHROMIUM_HI (PASSED): Retested this in Wasta Linux and it seems that after pasting the fonts in the text editor it shows the correct the output.

  

• TEST_CHROMIUM_KM (PASSED): Retested this in Wasta Linux and it is showing the correct output.

  

• TEST_CHROMIUM_KM_BS (PASSED): After Pressing Backspace key 5 times, the entire word is deleted.

[bharanidharanj]

SUITE_ANKI: Anki

GROUP_WASTA: Wasta 20.04 with Cinnamon

• TEST_ANKI_HI (PASSED): Retested this with Keyman 16.0.76-alpha build in Anki application and after pasting the fonts into the text edit, it shows the correct output.

  

[bharanidharanj]

SUITE_CHROMIUM: Chromium

GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11

• TEST_CHROMIUM_KO (PASSED): Retested this with Keyman 16.0.76-alpha in Ubuntu Focal Linux and it is showing the correct output.

  

• TEST_CHROMIUM_HI (PASSED): Retested this with Keyman 16.0.76-alpha in Ubuntu Focal Linux and it is showing the correct output.

  

[keyman-server]

Changes in this pull request will be available for download in Keyman version 16.0.78-alpha