search

keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
372 stars 102 forks source link

test(linux): Regression Tests for Keyman 17 - Alpha #8226

Closed MakaraSok closed 11 months ago

MakaraSok commented 1 year ago

Keyman for Linux Acceptance Test Procedures

  1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
  2. Copy these checklists of tests into a new issue (for tier transitions) or a comment on the PR.
  3. When annotating results, use > at the start of a new line under the checkbox to note the result. (For an example, see #3973).

General remark to the groups below

The login screen allows you to select different environments by clicking on the cog symbol. Different Ubuntu versions put it in different places and use slightly different icons. In the groups below the menu item to select is put in parenthesis and quotes, e.g. Ubuntu 18.04 Bionic with Gnome Shell and X11 ("Ubuntu").

Screenshot from 2022-06-09 15-35-23

Gather Assets for Testing

click to expand Either VM or physical machine will work. - [ ] Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu") - [ ] Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")
Expand to see how to do and verify this - when login in, click the cog and select "Ubuntu on Xorg" ![Screenshot from 2022-01-19 18-28-11](https://user-images.githubusercontent.com/181336/150183345-f07c7191-b12b-4739-b4fc-c585148db56c.png) - verify that you're using X11 by running the following command in a terminal: ```bash echo $XDG_SESSION_TYPE ``` This should output `x11`.
- [ ] Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")
Expand to see how to do and verify this - when login in, click the cog and select "Ubuntu on Xorg" ![Screenshot from 2022-01-19 18-28-11](https://user-images.githubusercontent.com/181336/150183345-f07c7191-b12b-4739-b4fc-c585148db56c.png) - verify that you're using X11 by running the following command in a terminal: ```bash echo $XDG_SESSION_TYPE ``` This should output `x11`.
- [ ] Wasta 20.04 with Cinnamon ("Cinnamon (Default)")

Setup Steps

click to expand - [ ] Install latest updates on the system: ```bash sudo apt update sudo apt --autoremove upgrade ``` - [ ] Uninstall previous versions of Keyman for Linux on the system: ```bash sudo apt --autoremove remove keyman ibus-keyman python3-keyman-config libkmnkbp0-0 ``` - [ ] Remove left-over artifacts by running: ```bash rm -rf ~/.local/share/keyman/ sudo rm -rf /usr/local/share/keyman/ ``` - [ ] Remove all keyboards from Input Sources that except "English (US)" - From Ubuntu, start "Settings" --> "Region & Language" (in newer Ubuntu versions: "Settings" --> "Keyboard") - On Wasta Linux, open IBus Keyboards instead --> Input Method tab - [ ] Add the ppa (for pre-beta tests, replace `keyman-beta` with `keyman-alpha`): ```bash sudo add-apt-repository ppa:keymanapp/keyman-beta sudo apt update ``` - [ ] Add a second non-Keyman keyboard so that the language/keyboard dropdown shows (these are the steps for Ubuntu; for Wasta they are similar):
Expand to see the steps - From Ubuntu, start "Settings" --> "Region & Language" (in newer Ubuntu versions: "Settings" --> "Keyboard"; on Wasta Linux open IBus Preferences which can be opened by typing "ibus keyboards"). - Below "Input Sources" click the "+" and then the three dots: ![Screenshot from 2022-02-03 11-46-53](https://user-images.githubusercontent.com/181336/152328230-8c70fe5a-32fb-4090-a50b-272300d82df7.png) - in the search field, type "French" (or any other language), "Other" and then of the French keyboards and finally the "Add" button. ![Screenshot from 2022-02-03 11-50-06](https://user-images.githubusercontent.com/181336/152328628-f578b3b3-250a-45f1-89de-f0bc8e1c43b6.png) - now the language/keyboard dropdown will show. If not, reboot. ![Screenshot from 2022-02-03 11-51-24](https://user-images.githubusercontent.com/181336/152329086-ede2657f-ad4d-4b17-bd1d-b24a8d3e3839.png)
- [ ] Reboot - [ ] Install Nanum fonts for typing Korean: ```bash sudo apt install fonts-nanum ``` - [ ] Change "Complex text layout" setting in LibreOffice Writer:
Expand to see the steps - In LO Tools/Options, go to the Language Settings/Languages tab. Under "Default Languages for Documents" check the box for "Asian" and select "Korean (RoK)" as language, and check "Complex text layout" and select "Khmer" as language. ![Screenshot from 2022-06-23 12-13-56](https://user-images.githubusercontent.com/181336/175276071-2fda6202-b2a2-498a-9638-517409e68dcb.png) - You'll also have to change the default style: go to Styles/Manage Styles, right-click on "Default Paragraph Style" and select Modify. Go to the Font tab. For Asian Text Font, select Family "NanumMyeongjo" and Language "Korean (RoK)". For CTL Font, select Family "Khmer Mondulkiri" (or another "Khmer" font) and Language "Khmer". ![Screenshot from 2022-06-23 12-11-35](https://user-images.githubusercontent.com/181336/175275721-d3b282e0-4efc-42b8-89aa-493468981141.png)

User Testing

click to expand ## SUITE_INSPECT: Visual Inspection of master/beta branches - **TEST_CI_MASTER**: Verify the master build for Linux reports green on - **TEST_CI_BETA**: Verify the beta build for Linux reports green on - **TEST_VERIFY_HISTORY**: Verify `/HISTORY.md` contains all the current changes ## SUITE_INSTALLATION: Basic installation - **GROUP_FOCAL**: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu") - **GROUP_JAMMY_X11**: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_KINETIC_X11**: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_WASTA**: Wasta 20.04 with Cinnamon ("Cinnamon (Default)") ### Tests - **TEST_INSTALL**: Install keyman and onboard - in a terminal window, run: ```bash sudo apt update sudo apt install keyman onboard-keyman ``` - verify that this works without showing any error - **TEST_START**: From the launcher, start "Keyman Configuration" - Verify "Keyman Configuration" panel appears - **TEST_KB_INSTALL**: Download and install a keyboard - Click "Download", enter search for "Khmer", and click the "Khmer Angkor" keyboard - Click "Install keyboard" - Verify that the Readme is shown in the "README" tab of the install window - Click "Install" button - Verify that the Welcome page appears after installation - Verify the new keyboard appears in "Keyman Configuration" - **TEST_KB_ADDED**: Verify the installed appears in the keyboard dropdown - Verify the language tag for the current keyboard appears in the keyboard/language dropdown list in the task bar (with language name (Khmer) and "Khmer Angkor" as the keyboard name) - From Ubuntu, start "Settings" --> "Region & Language" (in newer Ubuntu versions: "Settings" --> "Keyboard") - Verify the keyboard appears in the "Input Sources" list with the language name (Khmer) and keyboard name (Khmer Angkor) - **TEST_KB_WORKS**: Verify that typing with Keyman keyboard works in gedit - Type Windows-spacebar to switch to the Khmer Angkor keyboard - verify that the language tag "km" appears in the taskbar ![Screenshot from 2022-02-03 12-04-10](https://user-images.githubusercontent.com/181336/152330927-85322e71-5f48-4c35-8cf3-af0b69e2b447.png) - Open the text editor (gedit), verify that the newly installed keyboard can type - Open LibreOffice Writer, verify that the newly installed keyboard can type - Type Windows-spacebar to switch back to English keyboard - **TEST_KB_ICELANDIC**: Add a keyboard for an additional language - From the launcher, start "Keyman Configuration" - Verify "Keyman Configuration" dialog appears - Click "Download", search for "icelandic", and click the "EuroLatin (SIL) (Icelandic language)" keyboard - Click "Install keyboard" - Verify that the readme appears in the install window - Click "Install" button - Verify that the welcome file appears after installation - Verify the new keyboard appears in "Keyman Configuration" - Verify the new keyboard appears on the top right of the screen (similar to "Icelandic (EuroLatin (SIL)) - is") - Add the keyboard for an additional language in GNOME - From Ubuntu, start "Settings" --> "Region & Language" (in newer Ubuntu versions: "Settings" --> "Keyboard") - In "Input Sources", click "+" to add input source, click the three dots to get the search field - Search for German and select the "German (EuroLatin (SIL))" keyboard - Click "Add" - Verify the language/keyboard combination appears in the "Input Sources" list, similar to "German (EuroLatin (SIL))" - Verify the language tag for the new keyboard appears on the top right of the screen (similar to "German (EuroLatin (SIL)) - de") - Click Windows-spacebar to switch to the Icelandic keyboard - In a text editor (gedit), verify that the newly installed keyboard can type (e.g. typing `:a` should result in `ä`) - **TEST_KB_INST_PKG**: Install a package from disk - Open https://keyman.com/keyboards/sil_ipa in the browser. Download the `sil_ipa.kmp` keyboard linked after "Package Download". - In the "Keyman Configuration" dialog, click "Install" - Browse to the local `sil_ipa.kmp` file on disk and click "Open" to install it - Verify that the readme file appears in the README tab of the install window - Click "Install" button - Verify that the welcome file appears after installation - Verify the keyboard was added to GNOME - From Ubuntu, start "Settings" --> "Region & Language" (in newer Ubuntu versions: "Settings" --> "Keyboard") **Note:** you might have to close and re-open the Settings dialog if you still had it open to properly show the keyboards - Verify the keyboard appears in the "Input Sources" list (similar to "Undetermined (IPA SIL))") - Verify the language tag for the current keyboard appears on the top right of the screen - Click Windows-spacebar to switch to the IPA keyboard - In a text editor, verify that the newly installed keyboard can type: typing `n>` should result in `ŋ` ## SUITE_UI: Keyman Configuration dialog It is sufficient to run these tests on one platform only. - **TEST_UI_ABOUT:** About keyboard - In the "Keyman Configuration" dialog, select a keyboard and click "About" - Verify package and keyboard information is displayed - For a keyboard downloaded from keyman.com, verify a QR Code link is displayed - **TEST_UI_HELP:** Keyboard help - In the "Keyman Configuration" dialog, select a keyboard and click "Help" - If the package does not include a welcome file verify that the "Help" button is disabled - If the package includes a welcome file, verify that it appears. - **TEST_UI_OPTIONS:** Options - In the "Keyman Configuration" dialog, select the "EuroLatin (SIL)" keyboard and click "Options" - Verify the Options dialog appears. - Select a different keyboard that doesn't provide options - Verify the Options button is disabled. ## SUITE_OSK: On Screen Keyboard tests - **GROUP_FOCAL**: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu") - **GROUP_JAMMY_X11**: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_KINETIC_X11**: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_WASTA**: Wasta 20.04 with Cinnamon ("Cinnamon (Default)") ### Tests - **TEST_OSK:** On Screen Keyboard tests - From the launcher, start "onboard" - Verify the OSK for the current keyboard appears at the bottom of the screen - Verify that the default OSK (en-US) is shown if the current keyboard doesn't have one - Click Windows-spacebar to switch to the Khmer Angkor keyboard - Verify the OSK switches to match the Khmer Angkor keyboard - In a text editor (gedit), verify clicking on an OSK key produces expected output - Click Windows-spacebar and switch back to English keyboard - Verify the OSK switches back to the English keyboard - **TEST_OSK_CONFIG:** On Screen Keyboard configuration - From the launcher, start "onboard settings" - Verify the "Onboard Preferences" panel appears ## SUITE_CLI: Command line tools It is sufficient to run these tests on one platform only. Run the following tests in a terminal window. ### km-package-get - **TEST_CLI_GET:** Run `km-package-get`. Verify that you get an error message. - **TEST_CLI_GET_ID:** Run `km-package-get hieroglyphic`. Verify the file `~/.cache/keyman/hieroglyphic.kmp` got downloaded. - **TEST_CLI_GET_MAN:** Run `man km-package-get`. Verify it shows the manual page explaining the parameters. - **TEST_CLI_GET_WEB:** Open in the browser. Verify you get the same information as in the manual page. ### km-package-install - **TEST_CLI_INST:** Run `km-package-install`. Verify that you get an error message. - **TEST_CLI_INST_PKG:** Run `km-package-install -p sil_korda_jamo`. Verify that this adds the "Korean KORDA Jamo (SIL)" keyboard to the keyboard dropdown. - **TEST_CLI_INST_SHARED:** Run `sudo km-package-install -s -f ~/.cache/keyman/hieroglyphic.kmp`. Verify that this adds the "Hieroglyphic" keyboard for ancient Egyptian to the keyboard dropdown. - **TEST_CLI_INST_COMPL:** Type `km-package-install -p gff_amha` and press `TAB` key. Verify that this completes the command to `km-package-install -p gff_amharic`. - **TEST_CLI_INST_MAN:** Run `man km-package-install`. Verify it shows the manual page explaining the parameters. - **TEST_CLI_INST_WEB:** Open in the browser. Verify you get the same information as in the manual page. ### km-package-list-installed - **TEST_CLI_LIST:** Run `km-package-list-installed`. Verify that it lists the keyboards you previously installed. - **TEST_CLI_LIST_USER:** Run `km-package-list-installed -u`. Verify you get the same list as in the previous test. - **TEST_CLI_LIST_LONG:** Run `km-package-list-installed --long`. Verify that it lists the keyboards you previously installed, displaying details for each keyboard. - **TEST_CLI_LIST_SHARED:** Run `km-package-list-installed -s`. This should show the Hieroglyphic keyboard as the only one under "Installed shared Keyman keyboard packages". - **TEST_CLI_LIST_OS:** Run `km-package-list-installed -o`. There should be no keyboards under "Installed OS Keyman keyboard packages". - **TEST_CLI_LIST_MAN:** Run `man km-package-list-installed`. Verify it shows the manual page explaining the parameters. - **TEST_CLI_LIST_WEB:** Open in the browser. Verify you get the same information as in the manual page. ### km-kvk2ldml - **TEST_CLI_LDML:** Run `km-kvk2ldml`. Verify that you get an error message. - **TEST_CLI_LDML_FILE:** Run `km-kvk2ldml -o /tmp/test.ldml ~/.local/share/keyman/khmer_angkor/khmer_angkor.kvk`. Verify that this creates the file `/tmp/test.ldml`. - **TEST_CLI_LDML_PRNT:** Run `km-kvk2ldml -p ~/.local/share/keyman/khmer_angkor/khmer_angkor.kvk`. Verify that this prints information about the keyboard. - **TEST_CLI_LDML_KEYS:** Run `km-kvk2ldml -k -p ~/.local/share/keyman/khmer_angkor/khmer_angkor.kvk`. Verify that in addition to the information about the keyboard it prints the keys contained in the keyboard. - **TEST_CLI_LDML_MAN:** Run `man km-kvk2ldml`. Verify it shows the manual page explaining the parameters. - **TEST_CLI_LDML_WEB:** Open in the browser. Verify you get the same information as in the manual page. ### km-package-uninstall - **TEST_CLI_UNINST:** Run `km-package-uninstall`. Verify that you get an error message. - **TEST_CLI_UNINST_PKG:** Run `km-package-uninstall sil_euro_latin` - Verify that this removes the "Icelandic (EuroLatin (SIL))" as well as the "German (EuroLatin (SIL))" keyboards from the keyboard dropdown - Verify that the folder `~/.local/share/keyman/sil_euro_latin` no longer exists - **TEST_CLI_UNINST_SHARED:** Run `sudo km-package-uninstall -s hieroglyphic` - Verify that this removes the "Hieroglyphic" keyboard from the keyboard dropdown - Verify that the folder `/usr/local/share/keyman/hieroglyphic` no longer exists - **TEST_CLI_UNINST_MAN:** Run `man km-package-uninstall`. Verify it shows the manual page explaining the parameters. - **TEST_CLI_UNINST_WEB:** Open in the browser. Verify you get the same information as in the manual page. ### km-config - **TEST_CLI_CFG:** Run `km-config`. Verify that it opens the "Keyman Configuration" dialog. - **TEST_CLI_CFG_VER:** Run `km-config --version`. Verify that it outputs the version number. - **TEST_CLI_CFG_INST1:** Run `km-config -i file://$HOME/.cache/keyman/hieroglyphic.kmp`. Verify that this opens the installation dialog for the hieroglyphic keyboard. - **TEST_CLI_CFG_INST2:** Run `km-package-get el_dinka && km-config --install ~/.cache/keyman/el_dinka.kmp`. Verify that this opens the installation dialog for the el_dinka/Thuɔŋjäŋ keyboard. - **TEST_CLI_CFG_PKG:** Run `km-config keyman://download/keyboard/sil_pan_africa_positional?bcp47=bjt-latn` - Verify that this brings up the installation dialog for the "Pan Africa Positional (SIL)" keyboard - Click "Install" button - Verify that the "Pan Africa Positional (SIL)" shows up for Balanta-Ganja in the keyboards dropdown list - **TEST_CLI_CFG_MAN:** Run `man km-config`. Verify it shows the manual page explaining the parameters. - **TEST_CLI_CFG_WEB:** Open in the browser. Verify you get the same information as in the manual page. ## SUITE_KBDS: Verify specific keyboards - **GROUP_FOCAL**: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu") - **GROUP_JAMMY_X11**: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_KINETIC_X11**: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_WASTA**: Wasta 20.04 with Cinnamon ("Cinnamon (Default)") ### Tests - **TEST_IPA_WRITER:** Open LibreOffice Writer. Switch to "IPA (SIL)" keyboard. Type `n>`. Verify that the result is "ŋ". - **TEST_IPA_GEDIT:** Open gedit. Switch to "IPA (SIL)" keyboard. Type `n>`. Verify that the result is "ŋ". - **TEST_KO_WRITER:** Open LibreOffice Writer. Switch to "Korean KORDA Jamo (SIL)" keyboard. Type `hangeul`. Verify that the result is "한글". - **TEST_KO_GEDIT:** Open gedit. Switch to "Korean KORDA Jamo (SIL)" keyboard. Type `hangeul`. Verify that the result is "한글". - **TEST_KM_WRITER:** Open LibreOffice Writer. Switch to "Khmer Angkor" keyboard. Type `xEjmr`. Verify that the output is "ខ្មែរ". - **TEST_KM_GEDIT:** Open gedit. Switch to "Khmer Angkor" keyboard. Type `xEjmr`. Verify that the output is "ខ្មែរ". - **TEST_CONTEXT:** - Open gedit - Switch to "IPA (SIL)" keyboard. - Type ``` an m ``` - click after `n` - type `>` - verify that the output is ``` aŋ m ``` ## SUITE_UNINSTALL: Uninstallation - **GROUP_FOCAL**: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu") - **GROUP_JAMMY_X11**: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_KINETIC_X11**: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_WASTA**: Wasta 20.04 with Cinnamon ("Cinnamon (Default)") ### Sanity check Run `apt list python3-keyman-config`. The output should contain `[installed,automatic]`. If it's just `[installed]`, then go back to the [Setup Steps](#setup-steps) and uninstall Keyman. Afterwards run `sudo apt install keyman` before running the following tests. ### Tests - **TEST_UNINSTALL:** Remove the keyman package ```bash sudo apt --autoremove remove keyman ``` - Verify no errors are displayed - Verify the directory and files in `/usr/lib/python3/dist-packages/keyman_config` are gone - Verify that the user installed keyboards didn't get removed, i.e. the subdirectories under `~/.local/share/keyman` still exist and still contain the `*.kmx` and other files. ## SUITE_UPDATE: Update an existing installation - **GROUP_FOCAL**: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu") - **GROUP_KINETIC_X11**: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg") - **GROUP_WASTA**: Wasta 20.04 with Cinnamon ("Cinnamon (Default)") ### Preparation - remove keyman package ```bash sudo apt --autoremove remove keyman ``` - remove beta and alpha ppas: ```bash sudo rm /etc/apt/sources.list.d/keymanapp-ubuntu-keyman* ``` - add stable ppa ```bash sudo add-apt-repository ppa:keymanapp/keyman ``` - install latest stable version ```bash sudo apt update sudo apt install keyman onboard-keyman ``` - verify that this installed the latest stable version: `km-config --version` - re-add beta ppa: ```bash sudo add-apt-repository ppa:keymanapp/keyman-beta ``` For pre-beta tests install the alpha ppa: ```bash sudo add-apt-repository ppa:keymanapp/keyman-alpha ``` - reboot ### Tests - **TEST_UPGRADE**: Upgrade Keyman - Run the following command: ```bash sudo apt update sudo apt upgrade ``` - Verify that this installed the latest beta (alpha) version: `km-config --version` - verify that the Keyman keyboards still show up in the keyboards dropdown - switch to one of the Keyman keyboards and verify that typing with that keyboard still works ## What's New Tests ??? - Refer to the new changes in `/history.md` and verify functionality
keymanapp-test-bot[bot] commented 1 year ago

User Test Results

Test specification and instructions

✅ SUITE_INSPECT: Visual Inspection of master/beta branches

3 tests in 1 groups PASSED - ✅ **TEST_CI_MASTER ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1432961206))**: Verified that the master build for linux reports green on status.keyman site. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1432961206)) - ✅ **TEST_CI_BETA ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1432961206))**: Verified that the beta build for linux reports green on status.keyman site. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1432961206)) - ✅ **TEST_VERIFY_HISTORY ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1432961206))**: Verified that the History file contains all the current changes. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1432961206))

🟩 SUITE_INSTALLATION: Basic installation

✅ SUITE_UI: Keyman Configuration dialog

3 tests in 1 groups PASSED - ✅ **TEST_UI_ABOUT ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439800931))**: Tested with the latest Keyman 17.0.53-alpha build and it is correctly showing the About page. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439800931)) - ✅ **TEST_UI_HELP ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439800931))**: Clicking on the Help button opens the Help page correctly. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439800931)) - ✅ **TEST_UI_OPTIONS ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439800931))**: Clicking on the Options button opens the Options dialog correctly. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439800931))

✅ SUITE_OSK: On Screen Keyboard tests

8 tests in 4 groups PASSED * ✅ GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")
2 tests PASSED - ✅ **TEST_OSK ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451545631))**: Tested with the latest Keyman 17.0.59-alpha build in Ubuntu Focal Linux OS (VM) and here is my observation: Typing Khmer text in the OSK outputs the expected result in the text editor. Seems to be working fine. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451545631)) - ✅ **TEST_OSK_CONFIG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451545631))**: Running command shows the panel on the screen. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451545631))
* ✅ GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")
2 tests PASSED - ✅ **TEST_OSK ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439877799))**: Verified that the OSK seems to be working fine . ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439877799)) - ✅ **TEST_OSK_CONFIG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439877799))**: Able to see the OSK panel. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439877799))
* ✅ GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")
2 tests PASSED - ✅ **TEST_OSK ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451550430))**: Tested with the latest Keyman 17.0.59-alpha build in Ubuntu Kinetic Kudu Linux OS (VM) and here is my observation: Typing Khmer text in the OSK outputs the expected result in the text editor. Seems to be working fine. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451550430)) - ✅ **TEST_OSK_CONFIG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440037403))**: Able to see the OSK panel. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440037403))
* ✅ GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")
2 tests PASSED - ✅ **TEST_OSK ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439966841))**: Verified that the OSK is working fine . ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439966841)) - ✅ **TEST_OSK_CONFIG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439966841))**: Able to see the OSK panel. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1439966841))

✅ SUITE_CLI: Command line tools

35 tests in 1 groups PASSED - ✅ **TEST_CLI_GET ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528))**: Tested with Keyman 17.0.49-alpha build in Jammy Jelly Fish Linux OS 22.04 and it is showing the error message. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528)) - ✅ **TEST_CLI_GET_ID ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528))**: Run the given command and got the .kmp file downloaded. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528)) - ✅ **TEST_CLI_GET_MAN ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528))**: Verified that after running the given command it is showing the manual page that explaining the parameters. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528)) - ✅ **TEST_CLI_GET_WEB ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528))**: Opened the given browser link and verified that the same information appears in the manual page. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440105528)) - ✅ **TEST_CLI_INST ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157))**: Verified that it is showing the error message. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157)) - ✅ **TEST_CLI_INST_PKG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157))**: Verified that it adds Korean KORDA Jamo (SIL) keyboard to the keyboard dropdown. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157)) - ✅ **TEST_CLI_INST_SHARED ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157))**: Verified that by running the given command it adds Hieroglyphic keyboard in the keyboard dropdown. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157)) - ✅ **TEST_CLI_INST_COMPL ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157))**: Verified that pressing Tab key completes the command line. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157)) - ✅ **TEST_CLI_INST_MAN ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157))**: Verified that it is showing the manual page explaining the parameters. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157)) - ✅ **TEST_CLI_INST_WEB ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157))**: Verified that the given browser link shows the same information in the manual page. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1440174157)) - ✅ **TEST_CLI_LIST ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: (Tested in Keyman 17.0.59-alpha build) Verified that it lists the keyboards I previously installed. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LIST_USER ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: Verified that it shows the same list as in the previous test. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LIST_LONG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: Verified that it lists the keyboards I previously installed displaying details for each keyboard. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LIST_SHARED ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: Verified that "Installed shared Keyman keyboard packages" messages appeared in the terminal, since the Hieroglyphic keyboard is installed in the system. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LIST_OS ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: Verified that there is no keyboard lists under "Installed OS Keyman keyboard packages". ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LIST_MAN ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: Verified that it is showing manual page explaining the parameters. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LIST_WEB ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739))**: Verified that I am getting the same information when I opened the link in the browser. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451680739)) - ✅ **TEST_CLI_LDML ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011))**: Verified that it throws an error message, after running the mentioned command. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011)) - ✅ **TEST_CLI_LDML_FILE ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011))**: Verified that a test.ldml file was created after running the mentioned command. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011)) - ✅ **TEST_CLI_LDML_PRNT ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011))**: Verified that an information about the keyboard was printed on the terminal. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011)) - ✅ **TEST_CLI_LDML_KEYS ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011))**: Verified that it also printed the keys contained in the keyboard. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011)) - ✅ **TEST_CLI_LDML_MAN ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011))**: Verified that it is showing manual page explaining the parameters. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011)) - ✅ **TEST_CLI_LDML_WEB ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011))**: Verified that I am getting the same information when I opened the link in the browser. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451722011)) - ✅ **TEST_CLI_UNINST ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009))**: Verified that it throws an error message. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009)) - ✅ **TEST_CLI_UNINST_PKG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009))**: Verified that the sil_euro_latin keyboard has been removed from ~/.local/share/keyman folder. - ✅ **TEST_CLI_UNINST_SHARED ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009))**: Verified that the Hieroglyphic keyboard has been removed from /usr/local/share/keyman folder. - ✅ **TEST_CLI_UNINST_MAN ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009))**: Verified that it is showing manual page explaining the parameters. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009)) - ✅ **TEST_CLI_UNINST_WEB ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009))**: Verified that I am getting the same information when I opened the link in the browser. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451753009)) - ✅ **TEST_CLI_CFG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that it opens the "Keyman Configuration" dialog. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787)) - ✅ **TEST_CLI_CFG_VER ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that it outputs the Keyman Version number. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787)) - ✅ **TEST_CLI_CFG_INST1 ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that this opens the installation dialog for hieroglyphic keyboard. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787)) - ✅ **TEST_CLI_CFG_INST2 ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that this opens the installation dialog for the el_dinka/Thuɔŋjäŋ keyboard. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787)) - ✅ **TEST_CLI_CFG_PKG ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that it shows up for Balanta-Ganja in the keyboards dropdown list. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787)) - ✅ **TEST_CLI_CFG_MAN ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that it opens the manual page explaining the parameters. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787)) - ✅ **TEST_CLI_CFG_WEB ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))**: Verified that the same information appears in the browser. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1451846787))

🟩 SUITE_KBDS: Verify specific keyboards

✅ SUITE_UNINSTALL: Uninstallation

4 tests in 4 groups PASSED * ✅ GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")
1 tests PASSED - ✅ **TEST_UNINSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453060308))**: Verified that there are no errors displayed after remove the keyman. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453060308))
* ✅ GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")
1 tests PASSED - ✅ **TEST_UNINSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453103716))**: Verified that there are no errors displayed after remove the keyman. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453103716))
* ✅ GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")
1 tests PASSED - ✅ **TEST_UNINSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453162790))**: Verified that there are no errors displayed after remove the keyman. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453162790))
* ✅ GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")
1 tests PASSED - ✅ **TEST_UNINSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453132536))**: Verified that there are no errors displayed after remove the keyman. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453132536))

✅ SUITE_UPDATE: Update an existing installation

3 tests in 3 groups PASSED * ✅ GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")
1 tests PASSED - ✅ **TEST_UPGRADE ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453263102))**: Verified that it is showing the stable version. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453263102))
* ✅ GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")
1 tests PASSED - ✅ **TEST_UPGRADE ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453233507))**: Verified that it is showing the stable version. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453233507))
* ✅ GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")
1 tests PASSED - ✅ **TEST_UPGRADE ([PASSED](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453341537))**: Verified that it is showing the stable version. ([notes](https://github.com/keymanapp/keyman/issues/8226#issuecomment-1453341537))
bharanidharanj commented 1 year ago

SUITE_INSPECT: Visual Inspection of master/beta branches

bharanidharanj commented 1 year ago

SUITE_INSTALLATION: Basic installation

GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")

bharanidharanj commented 1 year ago

SUITE_INSTALLATION: Basic installation

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_INSTALLATION: Basic installation

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_INSTALLATION: Basic installation

GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")

bharanidharanj commented 1 year ago

SUITE_UI: Keyman Configuration dialog

bharanidharanj commented 1 year ago

SUITE_OSK: On Screen Keyboard tests

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_OSK: On Screen Keyboard tests

GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")

bharanidharanj commented 1 year ago

SUITE_OSK: On Screen Keyboard tests

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_CLI: Command line tools

bharanidharanj commented 1 year ago

SUITE_CLI: Command line tools

ermshiperete commented 1 year ago

@bharanidharanj I'm wondering how you managed to install the onboard-keyman package :smile: But this made me realize that I forgot to add onboard-keyman to Kinetic. That happened now, so you can test again.

@keymanapp-test-bot retest SUITE_INSTALLATION GROUP_KINETIC_X11 TEST_KB_ICELANDIC SUITE_OSK GROUP_KINETIC_X11 TEST_OSK

bharanidharanj commented 1 year ago

@ermshiperete Okay, I will do it.

bharanidharanj commented 1 year ago

SUITE_OSK: On Screen Keyboard tests

GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")

bharanidharanj commented 1 year ago

SUITE_OSK: On Screen Keyboard tests

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_INSTALLATION: Basic installation

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_CLI: Command line tools

bharanidharanj commented 1 year ago

SUITE_CLI: Command line tools

bharanidharanj commented 1 year ago

SUITE_CLI: Command line tools

bharanidharanj commented 1 year ago

SUITE_CLI: Command line tools

bharanidharanj commented 1 year ago

SUITE_KBDS: Verify specific keyboards

GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")

bharanidharanj commented 1 year ago

SUITE_KBDS: Verify specific keyboards

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_KBDS: Verify specific keyboards

GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")

bharanidharanj commented 1 year ago

SUITE_KBDS: Verify specific keyboards

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

bharanidharanj commented 1 year ago

SUITE_UNINSTALL: Uninstallation

GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")

Verified that the keyman_config files are gone from /usr/lib/python3/dist-packages folder.

Verified that the *.kmx and other files still exists in the ~/.local/share/keyman folder.

bharanidharanj commented 1 year ago

SUITE_UNINSTALL: Uninstallation

GROUP_JAMMY_X11: Ubuntu 22.04 Jammy with Gnome Shell and X11 ("Ubuntu on Xorg")

..sanity check

..no errors after uninstall

Verified that the keyman_config files are gone from /usr/lib/python3/dist-packages folder.

Verified that the *.kmx and other files still exists in the ~/.local/share/keyman folder.

bharanidharanj commented 1 year ago

SUITE_UNINSTALL: Uninstallation

GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")

Verified that the keyman_config files are gone from /usr/lib/python3/dist-packages folder.

Verified that the *.kmx and other files still exists in the ~/.local/share/keyman folder.

bharanidharanj commented 1 year ago

Test Results

SUITE_UNINSTALL: Uninstallation

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

Verified that the keyman_config files are gone from /usr/lib/python3/dist-packages folder.

Verified that the *.kmx and other files still exists in the ~/.local/share/keyman folder.

bharanidharanj commented 1 year ago

SUITE_UPDATE: Update an existing installation

GROUP_KINETIC_X11: Ubuntu 22.10 Kinetic with Gnome Shell and X11 ("Ubuntu on Xorg")

Verified that it is showing the latest alpha version.

Verified that the Keyman Keyboards are shows up in the keyboards dropdown and it is possible to switch to Keyman keyboard. Able to type with that keyboard in the text editor.

bharanidharanj commented 1 year ago

SUITE_UPDATE: Update an existing installation

GROUP_FOCAL: Ubuntu 20.04 Focal with Gnome Shell and X11 ("Ubuntu")

bharanidharanj commented 1 year ago

Test Results

SUITE_UPDATE: Update an existing installation

GROUP_WASTA: Wasta 20.04 with Cinnamon ("Cinnamon (Default)")

Verified that it is showing the latest alpha version.

Verified that the Keyman Keyboards are shows up in the keyboards dropdown and it is possible to switch to Keyman keyman keyboard. Able to type with that keyboard in the text editor.

ermshiperete commented 11 months ago

SUITE_KBDS: GROUP_JAMMY_X11: TEST_KM_WRITER (SKIPPED)

This test will show up correctly if the right font is used (Khmer Mondulkiri).

ermshiperete commented 11 months ago

I created #8896 for the failing test.

SUITE_INSTALLATION: GROUP_KINETIC_X11: TEST_KB_ICELANDIC(SKIPPED)