search

keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
398 stars 112 forks source link

test(windows): Keyman for Windows Acceptance Test Procedures #6377

Closed MakaraSok closed 2 years ago

MakaraSok commented 2 years ago

Keyman for Windows Acceptance Test Procedures

  1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.

  2. Results are annotated with results, use > at the start of a new line under the checkbox/Test to note the result.

User Testing

Gather Assets for Testing

Setup Steps

  1. Uninstall previous versions of Keyman Desktop and Keyman Developer on the system
  2. Restart system, install updates
  3. Make sure all additional Windows languages are removed
  4. Make sure no debug flags are set in HKCU/Software/Keyman/Debug

SUITE_INSPECT: Visual Inspection of master/beta branch

SUITE_BASELINE: Keyman for Windows Base Line Tests

Test cases

click to expand - **TEST_INSTALL**: - Install test build keyman-a.b.c.exe - Make sure Keyman is started and visible in the Notification area - **TEST_KEYBOARD_INSTALLATION_REMOTE**: - Start Keyman Configuration - Click Download Keyboard, select a keyboard from the download dialog and install it - If the package includes a readme file, verify that it appears in the install window. - If the package includes a welcome file, verify that it appears after installation. - Click OK to close Keyman Configuration - Verify that the newly installed keyboard appears in the Keyman menu and can be selected - **TEST_INSTALL_PKG_DISK**: - Install a package from disk ## - Locate a .kmp file on disk and double-click to install it - If the package includes a readme file, verify that it appears in the install window. - If the package includes a welcome file, verify that it appears after installation. - Verify that the keyboard appears in the Keyman menu - **TEST_KEYBOARD_OUTPUT**: - Start Notepad and select a Keyman keyboard (testing Legacy integration) - Verify that various test sequences are output correctly - Start Word and select a Keyman keyboard (testing TSF integration) - Verify that various test sequences are output correctly - **TEST_ON_SCREEN_KEYBOARD**: - Click Keyman icon in Notification area, select On Screen Keyboard - Verify that OSK displays correctly for selected Keyman keyboard - Verify that typing characters by clicking on OSK produces expected output

SUITE_CAPSLOCK:

Caps Lock

The test keyboard layouts are found in the keyman repo at app/windows/src/test/manual-tests/caps-lock-stores. There is a project file for the 3 keyboards used in this test. The project file can be used to build the keyboard packages required for the following tests.

The test cases below expect the usage of the capslock.kmp keyboard. That keyboard outputs pass or fail if following the test cases.

Prerequisites before each test

Test cases

click to expand - **TEST_CAPSLOCK-1**: uppercase with virtual key - press and release `a` Expected result: - `pass.` (with other keyboards uppercase `A`) - **TEST_CAPSLOCK-2**: lowercase with virtual key - press and hold 'Shift' - press and release `b` - release `Shift` Expected result: - `pass.` (with other keyboards lowercase `b`) - **TEST_CAPSLOCK-3**: capslock ignored for numbers - press and hold 'Shift' - press and release `3` - release `Shift` Expected result: - `pass.` (with other keyboards `#`) - **TEST_CAPSLOCK-4**: uppercase - press and release `c` Expected result: - `pass.` (with other keyboards uppercase `C`) - **TEST_CAPSLOCK-5**: lowercase - press and hold 'Shift' - press and release `d` - release `Shift` Expected result: - `pass.` (with other keyboards lowercase `d`)

CapsAlwaysOff

For these tests, use a keyboard with the CapsAlwaysOff store set. We call this keyboard capsalwaysoff below.

Any keyboard with that store set will work; if you don't have one at hand you can use the caps_always_off.kmp keyboard. The caps_always_off.kmp keyboard will prevent switching caps lock on. As a sanity check to verify that Keyman is actually active, pressing the key a will output ncaps_little_a, and Shift+a will output ncaps_shift_A.

Note: When testing in a virtual machine, use an on-screen keyboard (in VirtualBox: Input/Keyboard/Soft Keyboard) and observe the caps lock indicator of the on-screen keyboard. Using the hardware keyboard might show side effects with caps lock.

Prerequisites before each test

Test cases

click to expand - **TEST_CAPSOFF-1**: sanity check - switch to capsalwaysoff keyboard - press and release `a` Expected result: - output: `ncaps_little_a` - **TEST_CAPSOFF-2**: caps lock stays off - switch to capsalwaysoff keyboard - press and release `CapsLock` key - press and release `a` Expected result: - caps lock indicator stays turned off - output: `ncaps_little_a` - **TEST_CAPSOFF-3**: no caps lock while holding capslock key - switch to capsalwaysoff keyboard - press and hold `CapsLock` key - press and release `a` - release `CapsLock` key Expected result: - output: `ncaps_little_a` - **TEST_CAPSOFF-4**: no caps lock while holding capslock key - switch to capsalwaysoff keyboard - press and hold `CapsLock` key - press and hold `Shift` key - press and release `a` - release `CapsLock` and `Shift` keys Expected result: - output: `ncaps_shift_A` - **TEST_CAPSOFF-5**: switching turns off caps lock - turn on caps lock - switch to capsalwaysoff keyboard - press and release `a` Expected result: - caps lock indicator turned off - output: `ncaps_little_a`

SHIFT: CapsOnOnly/ShiftFreesCaps

For these tests, use a keyboard with the CapsOnOnly and ShiftFreesCaps stores set. We call this keyboard _shift_freescaps below.

Any keyboard with these stores set will work; if you don't have one at hand you can use the shift_frees_caps.kmp keyboard.

The _shift_freescaps.kmp keyboard will enable caps lock by pressing the CapsLock key, and will turn capslock off by pressing the Shift key. The keyboard outputs pass or fail if following the test cases.

Note: When testing in a virtual machine, use an on-screen keyboard (in VirtualBox: Input/Keyboard/Soft Keyboard) and observe the caps lock indicator of the on-screen keyboard. Using the hardware keyboard might show side effects with caps lock.

Prerequisites before each test

Test cases

click to expand - **TEST_CAPSONLY-1**: no caps - press and release `1` Expected result: - output: `pass.` - **TEST_CAPSONLY-2**: caps - press and release `CapsLock` - press and release `2` Expected result: - caps lock indicator turned on - output: `pass.` - **TEST_CAPSONLY-3**: caps doesn't toggle - press and release `CapsLock` - press and release `CapsLock` - press and release `6` Expected result: - caps lock indicator turned on - output: `pass.` - **TEST_CAPSONLY-4**: shift turns off - press and release `CapsLock` - press and hold `Shift` - press and release `3` - release `Shift` Expected result: - caps lock indicator turned off - output: `pass.` - **TEST_CAPSONLY-5**: shift by itself turns off - press and release `CapsLock` - press and release `Shift` Expected result: - caps lock indicator turned off - (no output)

SUITE_TSF_APP:

Type in-app that uses the windows TSF (MS Word, Fieldworks windows search field on the taskbar).

To run these tests the EuroLatin (SIL) Keyboard from here.

Test cases: double processing

SUITE_IMX_KEYBOARDS

Test IMX keyboards still work after core integration. see tests suite on #5936 and also #6187

This is the keyboard to be tested. imsample.zip If an IMTest keyboard is already installed you will need to uninstall it first, restart windows and then install the Imsample keyboard attached to this PR.

I have copied the tests from #5936 and made a few modifications.
The overall test we are aiming for is that in doing the tests below the keyboard does not cause a keyman or app crash.

Test cases

click to expand #### IMSAMPLE_KEYBOARD_INLINE_MENU: This keyboard uses the letters `aeom` to allow IMX input. This keyboard is a bit flakey being a demo and not polished. It also takes some setting up the following registry keys need to be added and set once installed. The location of the registry keys are at. `Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine\Active Keyboards\imsample` and are of type REG_DWORD The keys are `ShowIMWindow` and `ShowIMWindowAlways` they both need to be set to 0. In this mode IM window is not shown rather the app text is updated with a menu when either `aeom` is pressed. For example when type `he` the text will become `h[1ɛ 2ɜ 3ə 4e 5ɘ]` then you type the number `3` the text displayed should now be `hə` * **TEST_IMSAMPLE_INPUT:** - Install Imsample keyboard attached to this PR. - Select the keyboard its name is `IMTest` - Type some keys that are not the special keys e.g. `t` `r` `y` - - Type the `space` key then `a` then type 1 to get the 1st option - Expected Result: `try æ` - continue to the next test * **TEST_IMSAMPLE_INPUT_CONT:** ... contd from above - Type 'm' and type 3 to get `###` option - Expected Result: `try æ###` * **TEST_IMSAMPLE_BACKSPACE:** Using Notepad or Libreoffice - Type `f` - Type `e` - The screen should have f[1ɛ 2ɜ 3ə 4e 5ɘ] - Now press Backspace. the whole option menu including the square brackets `[` `]` should be deleted. - Expected result `f` #### IMSAMPLE_KEYBOARD_IM_WINDOW: This keyboard uses the letters `aeom` to allow IMX input. This time a IM window should display. ![image](https://user-images.githubusercontent.com/58423624/157192172-2ba2d0e4-8917-49b3-9709-00d7e9014645.png) The first time you use the keyboard with an application the registry keys will be created and by default set to 1. If the IM window is not appearing you will need to verify the value of two registry keys. The location of the registry keys are at. `Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine\Active Keyboards\imsample` and are of type REG_DWORD The keys are `ShowIMWindow` and `ShowIMWindowAlways` they both need to be set to 1. Note for the IM window you need to use the mouse pointer to select the options, you can't type the number. * **TEST_IMSAMPLE_INPUT_IMW:** - Install Imsample keyboard attached to this PR (if not done already) - Select the keyboard its name is `IMTest` - Type some keys that are not the special keys e.g. `t` `r` `y` - Type the `space` key then `a` then click the 1st option - Expected Result: `try æ` - continue to the next test * **TEST_IMSAMPLE_INPUT_IMW_CONT:** ... contd from above - Type 'm' and select the 3rd `###` option - Expected Result: `try æ###` #### SIMPLIFIED_CHINESE: This keyboard will display the IMX window as soon as a string the matches the pinyin for one or more characters are typed. For all these tests install the Simplified Chinese Keyboard cs-pinyin.cmp found [here](https://keyman.com/keyboards/cs-pinyin) * **TEST_SIMPLIFIED_CHINESE_SINGLE**: Using Notepad or equivalent. - Type `o` - The IMX window should appear - Choose the 4th option - Expected result: 喔 * **TEST_SIMPLIFIED_CHINESE_MULTIPLE**: Using Notepad or equivalent. - Type `h` `a` `n` `z` `i` - The IMX window will appear and in the top left the letters `hanzi` will be present - Choose the 5th option - Expected result: 汉字变换 * **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1**: Using Notepad or equivalent. This is to test the backspace occurring mid pinyin sequence before a character is output to the app - Type `h` `a` `n` `z` `i` - The IMX window will appear and in the top left the letters `hanzi` will be present - Press Backspace twice - The IMX window should now have the top left letters of `han`. - Choose the 5th option - Expected result: 汗 * **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2**: Using Notepad or equivalent. This is to test the backspace of already output characters. - Type `h` `a` `n` `z` `i` - The IMX window will appear and in the top left the letters `hanzi` will be present. - Choose the 5th option - Expected result: 汉字变换 - Press Backspace twice - Expected result: 汉字
keymanapp-test-bot[bot] commented 2 years ago

User Test Results

Test specification and instructions

✅ SUITE_INSPECT: Visual Inspection of master/beta branch

2 tests in 1 groups PASSED - ✅ **TEST_CI_MASTER ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1081951286))**: Successfully installed the latest Master build. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1081951286)) - ✅ **TEST_CI_BETA ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1081951286))**: Successfully installed the latest Beta build. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1081951286))

✅ SUITE_BASELINE: Keyman for Windows Base Line Tests

25 tests in 5 groups PASSED * ✅ GROUP_WIN7_x86:
5 tests PASSED - ✅ **TEST_INSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: Successfully installed even though there is a UAC warning that Keyman is of an unknown publisher, i.e. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543)) - ✅ **TEST_KEYBOARD_INSTALLATION_REMOTE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_INSTALL_PKG_DISK ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_KEYBOARD_OUTPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_ON_SCREEN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found.
* ✅ GROUP_WIN7_x64:
5 tests PASSED - ✅ **TEST_INSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: Successfully installed. - ✅ **TEST_KEYBOARD_INSTALLATION_REMOTE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_INSTALL_PKG_DISK ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_KEYBOARD_OUTPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_ON_SCREEN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found.
* ✅ GROUP_WIN8_1:
5 tests PASSED - ✅ **TEST_INSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: Successfully installed. - ✅ **TEST_KEYBOARD_INSTALLATION_REMOTE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106216129))**: No issue found. - ✅ **TEST_INSTALL_PKG_DISK ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_KEYBOARD_OUTPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_ON_SCREEN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found.
* ✅ GROUP_WIN10:
5 tests PASSED - ✅ **TEST_INSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1081965027))**: Successfully installed the latest Keyman Beta 15.0.224 build in Windows 10 OS. - ✅ **TEST_KEYBOARD_INSTALLATION_REMOTE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1081965027))**: Successfully installed the Tamil Keyboard and it is working as expected. - ✅ **TEST_INSTALL_PKG_DISK ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1082761293))**: Tested this with the latest Keyman 5.224-beta build and it is working as expected. - ✅ **TEST_KEYBOARD_OUTPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1082761293))**: Tested this with the latest Keyman 5.224-beta build and it is working as expected. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1082761293)) - ✅ **TEST_ON_SCREEN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1082761293))**: Tested this with the latest Keyman 5.224-beta build and it is working as expected. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1082761293))
* ✅ GROUP_WIN11:
5 tests PASSED - ✅ **TEST_INSTALL ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: Successfully installed 15.0.234-beta on Windows 11 Pro 21H2 (OS build: 22000.593). - ✅ **TEST_KEYBOARD_INSTALLATION_REMOTE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_INSTALL_PKG_DISK ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: The keyboard gets installed from file and works as expected. - ✅ **TEST_KEYBOARD_OUTPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: The keyboard produces the correct/expected output in LibreOffice and Windows search box (note that no Word is available on this VM.) - ✅ **TEST_ON_SCREEN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: OSK works as expected.

✅ SUITE_CAPSLOCK:

75 tests in 5 groups PASSED * ✅ GROUP_WIN7_x86:
15 tests PASSED - ✅ **TEST_CAPSLOCK-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSLOCK-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSLOCK-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSLOCK-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSLOCK-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSOFF-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSOFF-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSOFF-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSOFF-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSOFF-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSONLY-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSONLY-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSONLY-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSONLY-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_CAPSONLY-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))**: The problem encountered in the previous test (see the quote below) is a known issue where the soft key behaves differently from the that on an actual device. This test case is now marked as PASSED for this reason. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))
* ✅ GROUP_WIN7_x64:
15 tests PASSED - ✅ **TEST_CAPSLOCK-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSLOCK-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSLOCK-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSLOCK-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSLOCK-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSOFF-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSOFF-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSOFF-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSOFF-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSOFF-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSONLY-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSONLY-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSONLY-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSONLY-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_CAPSONLY-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))**: The problem encountered in the previous test (see the quote below) is a known issue where the soft key behaves differently from the that on an actual device. This test case is now marked as PASSED for this reason. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))
* ✅ GROUP_WIN8_1:
15 tests PASSED - ✅ **TEST_CAPSLOCK-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSLOCK-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSLOCK-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSLOCK-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSLOCK-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSOFF-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSOFF-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSOFF-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSOFF-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSOFF-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSONLY-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSONLY-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSONLY-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSONLY-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_CAPSONLY-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))**: The problem encountered in the previous test (see the quote below) is a known issue where the soft key behaves differently from the that on an actual device. This test case is now marked as PASSED for this reason. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))
* ✅ GROUP_WIN10:
15 tests PASSED - ✅ **TEST_CAPSLOCK-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084582263))**: Tested this with the latest Keyman 15.0.224 along with CapsLock Keyboard and it is working as expected. - ✅ **TEST_CAPSLOCK-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1108400354))**: As per Makara's suggestion I have re-tested this with the latest Keyman 15.0.239-beta build in Windows 10 native OS and it is passed. It is better to mention that, which environment (Windows 10 Native machine or Virtual Machine) will this testing be executed. Thanks. :-) ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1108400354)) - ✅ **TEST_CAPSLOCK-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084582263))**: Tested this with the latest Keyman 15.0.224 along with CapsLock Keyboard and it is working as expected. - ✅ **TEST_CAPSLOCK-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084582263))**: Tested this with the latest Keyman 15.0.224 along with CapsLock Keyboard and it is working as expected. - ✅ **TEST_CAPSLOCK-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084582263))**: Tested this with the latest Keyman 15.0.224 along with CapsLock Keyboard and it is working as expected. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084582263)) - ✅ **TEST_CAPSOFF-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084558088))**: Tested this with the latest Keyman 15.224-beta build along with Caps_Always_Off Keyboard and it is working as expected - ✅ **TEST_CAPSOFF-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084558088))**: Tested this with the latest Keyman 15.224-beta build along with Caps_Always_Off Keyboard and it is working as expected - ✅ **TEST_CAPSOFF-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084558088))**: Tested this with the latest Keyman 15.224-beta build along with Caps_Always_Off Keyboard and it is working as expected - ✅ **TEST_CAPSOFF-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084558088))**: Tested this with the latest Keyman 15.224-beta build along with Caps_Always_Off Keyboard and it is working as expected - ✅ **TEST_CAPSOFF-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084558088))**: Tested this with the latest Keyman 15.224-beta build along with Caps_Always_Off Keyboard and it is working as expected ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084558088)) - ✅ **TEST_CAPSONLY-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084530615))**: Tested this with Keyman 15.0.224-beta build along with shift_frees_caps Keyboard and it is working as expected - ✅ **TEST_CAPSONLY-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084530615))**: Tested this with Keyman 15.0.224-beta build along with shift_frees_caps Keyboard and it is working as expected - ✅ **TEST_CAPSONLY-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084530615))**: Tested this with Keyman 15.0.224-beta build along with shift_frees_caps Keyboard and it is working as expected - ✅ **TEST_CAPSONLY-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084530615))**: Tested this with Keyman 15.0.224-beta build along with shift_frees_caps Keyboard and it is working as expected - ✅ **TEST_CAPSONLY-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084530615))**: Tested this with Keyman 15.0.224-beta build along with shift_frees_caps Keyboard and it is working as expected
* ✅ GROUP_WIN11:
15 tests PASSED - ✅ **TEST_CAPSLOCK-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSLOCK-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSLOCK-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSLOCK-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSLOCK-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSOFF-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSOFF-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSOFF-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSOFF-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSOFF-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSONLY-1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSONLY-2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSONLY-3 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSONLY-4 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101995699))**: No issue found. - ✅ **TEST_CAPSONLY-5 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))**: The problem encountered in the previous test (see the quote below) is a known issue where the soft key behaves differently from the that on an actual device. This test case is now marked as PASSED for this reason. ([notes](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106239115))

✅ SUITE_TSF_APP:

10 tests in 5 groups PASSED * ✅ GROUP_WIN7_x86:
2 tests PASSED - ✅ **TEST_DOUBLE_PROCESSING_SEARCHBAR ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_DOUBLE_PROCESSING_NOTEPAD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found.
* ✅ GROUP_WIN7_x64:
2 tests PASSED - ✅ **TEST_DOUBLE_PROCESSING_SEARCHBAR ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_DOUBLE_PROCESSING_NOTEPAD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found.
* ✅ GROUP_WIN8_1:
2 tests PASSED - ✅ **TEST_DOUBLE_PROCESSING_SEARCHBAR ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_DOUBLE_PROCESSING_NOTEPAD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found.
* ✅ GROUP_WIN10:
2 tests PASSED - ✅ **TEST_DOUBLE_PROCESSING_SEARCHBAR ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084629238))**: Tested this with the latest Keyman 15.0.224-beta build along with installed Eurolatin Keyboard and it is working as expected. - ✅ **TEST_DOUBLE_PROCESSING_NOTEPAD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1104793476))**: ?):** Even though Keyman seems to start very slowly, it doesn't prevent me from using the keyboard once it's loaded and ready to use.
* ✅ GROUP_WIN11:
2 tests PASSED - ✅ **TEST_DOUBLE_PROCESSING_SEARCHBAR ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: `abc` does show in the output after deleting the last character. - ✅ **TEST_DOUBLE_PROCESSING_NOTEPAD ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: `abcd` does show in the output in Notepad after Keyman got started and EuroLatin (SIL) keyboard was selected and used to type the sequence.

✅ SUITE_IMX_KEYBOARDS:

45 tests in 5 groups PASSED * ✅ GROUP_WIN7_x86:
9 tests PASSED - ✅ **TEST_IMSAMPLE_INPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_IMSAMPLE_BACKSPACE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_SINGLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_MULTIPLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1102392543))**: No issue found.
* ✅ GROUP_WIN7_x64:
9 tests PASSED - ✅ **TEST_IMSAMPLE_INPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_IMSAMPLE_BACKSPACE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_SINGLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_MULTIPLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103468036))**: No issue found.
* ✅ GROUP_WIN8_1:
9 tests PASSED - ✅ **TEST_IMSAMPLE_INPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_IMSAMPLE_BACKSPACE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_SINGLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_MULTIPLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1103738449))**: No issue found.
* ✅ GROUP_WIN10:
9 tests PASSED - ✅ **TEST_IMSAMPLE_INPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084681057))**: Tested this with the latest Keyman 15.0.224-beta along with IMSample test Keyboard as per the instructions and it is working as expected. - ✅ **TEST_IMSAMPLE_INPUT_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084681057))**: Tested this with the latest Keyman 15.0.224-beta along with IMSample test Keyboard as per the instructions and it is working as expected. - ✅ **TEST_IMSAMPLE_BACKSPACE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084681057))**: Tested this with the latest Keyman 15.0.224-beta along with IMSample test Keyboard as per the instructions and it is working as expected. - ✅ **TEST_IMSAMPLE_INPUT_IMW ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084681057))**: Tested this with the latest Keyman 15.0.224-beta along with IMSample test Keyboard as per the instructions and it is working as expected. - ✅ **TEST_IMSAMPLE_INPUT_IMW_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1084681057))**: Tested this with the latest Keyman 15.0.224-beta along with IMSample test Keyboard as per the instructions and it is working as expected. - ✅ **TEST_SIMPLIFIED_CHINESE_SINGLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106207964))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_MULTIPLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106207964))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106207964))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1106207964))**: No issue found.
* ✅ GROUP_WIN11:
9 tests PASSED - ✅ **TEST_IMSAMPLE_INPUT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_IMSAMPLE_BACKSPACE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_IMSAMPLE_INPUT_IMW_CONT ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_SINGLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_MULTIPLE ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found. - ✅ **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2 ([PASSED](https://github.com/keymanapp/keyman/issues/6377#issuecomment-1101276717))**: No issue found.
bharanidharanj commented 2 years ago

SUITE_INSPECT: Visual Inspection of master/beta branch

bharanidharanj commented 2 years ago

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN10:

bharanidharanj commented 2 years ago

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN10:

bharanidharanj commented 2 years ago

SUITE_CAPSLOCK:

GROUP_WIN10:

bharanidharanj commented 2 years ago

SUITE_CAPSLOCK:

GROUP_WIN10:

CAPSOFF - Results - in Notepad Editor

bharanidharanj commented 2 years ago

SUITE_CAPSLOCK:

GROUP_WIN10:

CapsLock Result in Text Editor:

bharanidharanj commented 2 years ago

SUITE_TSF_APP:

GROUP_WIN10:

bharanidharanj commented 2 years ago

SUITE_IMX_KEYBOARDS:

GROUP_WIN10:

MakaraSok commented 2 years ago

Test Results

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN11:

SUITE_TSF_APP:

GROUP_WIN11:

SUITE_IMX_KEYBOARDS:

GROUP_WIN11:

All tests done in Notepad.

MakaraSok commented 2 years ago

Test Results

SUITE_CAPSLOCK:

GROUP_WIN11:

MakaraSok commented 2 years ago

Test Results

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN7_x86:

Tested with Keyman 15.0.234-beta on Windows 7 Professional - 6.1.7601 Service Pack 1 Build 7601 - 32-bit OS

SUITE_CAPSLOCK:

GROUP_WIN7_x86:

SUITE_TSF_APP:

GROUP_WIN7_x86:

SUITE_IMX_KEYBOARDS:

GROUP_WIN7_x86:

MakaraSok commented 2 years ago

Test Results

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN7_x64:

Tested with Keyman 15.0.234-beta on Windows 7 Professional - 6.1.7601 Service Pack 1 Build 7601 - 64-bit OS

SUITE_CAPSLOCK:

GROUP_WIN7_x64:

SUITE_TSF_APP:

GROUP_WIN7_x64:

SUITE_IMX_KEYBOARDS:

GROUP_WIN7_x64:

MakaraSok commented 2 years ago

Test Results

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN8_1:

Tested with Keyman 15.0.234-beta on Windows 8.1 Pro - 64-bit OS

SUITE_CAPSLOCK:

GROUP_WIN8_1:

SUITE_TSF_APP:

GROUP_WIN8_1:

SUITE_IMX_KEYBOARDS:

GROUP_WIN8_1:

MakaraSok commented 2 years ago

The test set needs to be improved and updated in GH wiki page based on this Platform Acceptance Test for Windows · keymanapp:keyman Wiki.pdf

MakaraSok commented 2 years ago

Test Results

SUITE_TSF_APP:

Tested with Keyman 15.0.234-beta on Windows 10 Pro - x64 - 21H2

GROUP_WIN10:

SUITE_IMX_KEYBOARDS:

GROUP_WIN10:

https://user-images.githubusercontent.com/28331388/164392788-85849921-81ca-4193-9ef5-494166a31e67.mov

MakaraSok commented 2 years ago

Test Results

SUITE_IMX_KEYBOARDS:

Tested with Keyman 15.0.238-beta on Windows 10 Pro - x64 - 21H2

GROUP_WIN10:

MakaraSok commented 2 years ago

Test Results

SUITE_BASELINE: Keyman for Windows Base Line Tests

GROUP_WIN8_1:

Tested with Keyman 15.0.234-beta on Windows 8.1 Pro - 64-bit OS

MakaraSok commented 2 years ago

Test Results

SUITE_CAPSLOCK:

GROUP_WIN7_x86:

The indicators are not turned off after pressing the shift key.

GROUP_WIN7_x64:

The indicators are not turned off after pressing the shift key.

GROUP_WIN8_1:

The indicators are not turned off after pressing the shift key.

GROUP_WIN11:

The indicators are not turned off after pressing the shift key.

bharanidharanj commented 2 years ago

SUITE_CAPSLOCK:

GROUP_WIN10:

MakaraSok commented 2 years ago

All tests are OK after all.