Closed rc-swag closed 1 year ago
Test specification and instructions
✅ GROUP_WIN10:
🟥 GROUP_WIN11:
TEST_KEYBOARD_OUTPUT (FAILED): I noticed that the installed 'khmer angkor' keyboard was missing the in the Keyman menu even though it appears on the Keyman Configuration dialog. Seems to be an issue.
TEST_ON_SCREEN_KEYBOARD (PASSED): Verified that OSK displayed correctly for the selected keyboard. Able to type in the Notepad using OSK.
I think we can remove LPKEYBOARD Keyboard;
from tagINTKEYBOARDINFO
in keymanengine.h, along with deleting legacy_kmx_memory.h altogether, yes? And that should filter out a bunch of other obsolete bits and pieces I think? Should we also be removing all references to legacy_kmx_file.h?
addins.cpp/addins.h should also be deleteable I think
TEST_DOUBLE_PROCESSING_FIREFOX (PASSED): Verified that it shows the expected result abc
TEST_DOUBLE_PROCESSING_SEARCHBAR (PASSED): Verified that it shows the expected result abc
TEST_DOUBLE_PROCESSING_NOTEPAD (PASSED): Verified that it shows the expected result abcd
TEST_IMSAMPLE_INPUT (FAILED): Installed the imsample keyboard in Windows 11 OS (VM) . Noticed that after pressing 1 or choosing the 1 from the Keyman Sample IM window is showing the expected output on the Notepad. Seems to be an issue.
TEST_IMSAMPLE_INPUT_CONT (FAILED): Not able to choose the letters from the Keyman Sample IM Window.
TEST_IMSAMPLE_BACKSPACE (FAILED): Not able to choose the letters from the Keyman Sample IM Window.
TEST_IMSAMPLE_INPUT_IMW (FAILED): Not able to choose the letters from the Keyman Sample IM Window.
TEST_IMSAMPLE_INPUT_IMW_CONT (FAILED): Not able to choose the letters from the Keyman Sample IM Window.
Developer build failure can be ignored -- due to updated build.sh pack
command for local build of npm packages; this will be available once merged.
TEST_SIMPLIFIED_CHINESE_SINGLE (FAILED): I noticed that the result appears on the body of the notepad is "?" mark. But it correctly appears in the title bar of the notepad. Seems to be an issue.
TEST_SIMPLIFIED_CHINESE_MULTIPLE (FAILED): The result appears on the body of the notepad is "?" mark. But it correctly appears in the title bar of the notepad.
TEST_SIMPLIFIED_CHINESE_BACKSPACE_1 (FAILED): The result appears on the body of the notepad is "?" mark. But it correctly appears in the title bar of the notepad.
TEST_SIMPLIFIED_CHINESE_BACKSPACE_2 (FAILED): The result appears on the body of the notepad is "?" mark. But it correctly appears in the title bar of the notepad.
@bharanidharanj I have reproduced the failures you have found and will move this back to draft to investigate.
I made mistake when removing the addin.h/addin.c from the project. In the hook message for UNI_CHAR I should have simply done nothing. However, I moved to the text message. Regardless I have reverted the commit and created a separate PR just to remove addins for easier tracking. It can simply cherry-pick the commit but handle the message correctly.
@keymanapp-test-bot retest SUITE_BASELINE GROUP_WIN10 TEST_KEYBOARD_OUTPUT
In Investing the failed test for Windows 11 on SUITE_IMX_KEYBOARDS
it appears that these tests also fail for Keyman 14 and 16. Issue #8697 has been created to address this issue.
So the failing SUITE_IMX_KEYBOARDS
, GROUP_WIN11
will be addressed in issue #8697
Since @mcdurdin last approved it 3 commits have been made and it is now ready for review again.
In an attempt to keep this PR from getting harder to review, the following separate issues have been made.
The failing SUITE_IMX_KEYBOARDS
, GROUP_WIN11
will be addressed in issue #8697. This is not a result of this PR but rather discovered in testing this PR.
@bharanidharanj I have reproduced the failures you have found and will move this back to draft to investigate.
Okay, Thanks @rc-swag
TEST_KEYBOARD_OUTPUT (PASSED): Tested with the attached PR build (17.0.87-alpha-test-8593) in Windows 10 OS and here is my observation: 1. Able to see the Khmer keyboard in both Keyman Configuration dialog as well as in the Keyman menu list in the task bar. 2. Able to type in both the keyboard using Notepad app. Seems to be working fine.
The failed tests fail currently on Windows 11 with Keyman 15 and up. https://github.com/keymanapp/keyman/issues/8697 has been created to deal with this. Therefore can merge this PR.
Changes in this pull request will be available for download in Keyman version 17.0.109-alpha
Fixes: #5442
Removing the legacy code that was replaced by the core. Also removing the feature flag for core integration.
User Testing
This will require a full acceptance test just to insure coverage of all the functionality.
Keyman for Windows Acceptance Test Procedures
These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
Results are annotated with results, use
>
at the start of a new line under the checkbox/Test to note the result.User Testing
Gather Assets for Testing
app/windows/src/test/manual-tests/caps-lock-stores
.Setup Steps
SUITE_BASELINE: Keyman for Windows Base Line Tests
Test cases
click to expand
- **TEST_INSTALL**: - Install test build keyman-a.b.c.exe - Make sure Keyman is started and visible in the Notification area - **TEST_KEYBOARD_INSTALLATION_REMOTE**: - Start Keyman Configuration - Click Download Keyboard, select a keyboard from the download dialog and install it - If the package includes a readme file, verify that it appears in the install window. - If the package includes a welcome file, verify that it appears after installation. - Click OK to close Keyman Configuration - Start Notepad - Verify that the newly installed keyboard appears in the Keyman menu and can be selected - **TEST_INSTALL_PKG_DISK**: - Install a package from disk ## - Locate a .kmp file on disk and double-click to install it - If the package includes a readme file, verify that it appears in the install window. - If the package includes a welcome file, verify that it appears after installation. - Start Notepad - Verify that the keyboard appears in the Keyman menu - **TEST_KEYBOARD_OUTPUT**: - Start Notepad and select a Keyman keyboard (testing Legacy integration) - Verify that various test sequences are output correctly - Start Word and select a Keyman keyboard (testing TSF integration) - Verify that various test sequences are output correctly - **TEST_ON_SCREEN_KEYBOARD**: - Click Keyman icon in Notification area, select On Screen Keyboard - Verify that OSK displays correctly for selected Keyman keyboard - Verify that typing characters by clicking on OSK produces expected outputSUITE_CAPSLOCK:
Caps Lock
The test keyboard layouts are found in the keyman repo at
app/windows/src/test/manual-tests/caps-lock-stores
. There is a project file for the 3 keyboards used in this test. The project file can be used to build the keyboard packages, but you can conveniently use the.kmp
file zipped and included respectively below.The test cases below expect the usage of the
capslock.kmp.zip
keyboard. That keyboard outputs pass or fail if following the test cases.Prerequisites before each test
capslock.kmp
.Test cases
click to expand
- **TEST_CAPSLOCK-1**: uppercase with virtual key - press and release `a` Expected result: - `pass.` (with other keyboards uppercase `A`) - **TEST_CAPSLOCK-2**: lowercase with virtual key - press and hold 'Shift' - press and release `b` - release `Shift` Expected result: - `pass.` (with other keyboards lowercase `b`) - **TEST_CAPSLOCK-3**: capslock ignored for numbers - press and hold 'Shift' - press and release `3` - release `Shift` Expected result: - `pass.` (with other keyboards `#`) - **TEST_CAPSLOCK-4**: uppercase - press and release `c` Expected result: - `pass.` (with other keyboards uppercase `C`) - **TEST_CAPSLOCK-5**: lowercase - press and hold 'Shift' - press and release `d` - release `Shift` Expected result: - `pass.` (with other keyboards lowercase `d`)CapsAlwaysOff
For these tests, use a keyboard with the
caps_always_off.kmp.zip
store set. We call this keyboard capsalwaysoff below.Any keyboard with that store set will work; if you don't have one at hand you can use the
caps_always_off.kmp
keyboard. The caps_always_off.kmp keyboard will prevent switching caps lock on. As a sanity check to verify that Keyman is actually active, pressing the keya
will outputncaps_little_a
, andShift+a
will outputncaps_shift_A
.Note: When testing in a virtual machine, use an on-screen keyboard (in VirtualBox: Input/Keyboard/Soft Keyboard) and observe the caps lock indicator of the on-screen keyboard. Using the hardware keyboard might show side effects with caps lock.
Prerequisites before each test
CapsAlwaysOff
store set, e.g.caps_always_off.kmp
.Test cases
click to expand
- **TEST_CAPSOFF-1**: sanity check - switch to capsalwaysoff keyboard - press and release `a` Expected result: - output: `ncaps_little_a` - **TEST_CAPSOFF-2**: caps lock stays off - switch to capsalwaysoff keyboard - press and release `CapsLock` key - press and release `a` Expected result: - caps lock indicator stays turned off - output: `ncaps_little_a` - **TEST_CAPSOFF-3**: no caps lock while holding capslock key - switch to capsalwaysoff keyboard - press and hold `CapsLock` key - press and release `a` - release `CapsLock` key Expected result: - output: `ncaps_little_a` - **TEST_CAPSOFF-4**: no caps lock while holding capslock key - switch to capsalwaysoff keyboard - press and hold `CapsLock` key - press and hold `Shift` key - press and release `a` - release `CapsLock` and `Shift` keys Expected result: - output: `ncaps_shift_A` - **TEST_CAPSOFF-5**: switching turns off caps lock - turn on caps lock - switch to capsalwaysoff keyboard - press and release `a` Expected result: - caps lock indicator turned off - output: `ncaps_little_a`SHIFT: CapsOnOnly/ShiftFreesCaps
For these tests, use a keyboard with the
CapsOnOnly
andShiftFreesCaps
stores set. We call this keyboard _shift_freescaps below.Any keyboard with these stores set will work; if you don't have one at hand you can use the
shift_frees_caps.kmp.zip
keyboard.The _shift_freescaps.kmp keyboard will enable caps lock by pressing the
CapsLock
key, and will turn capslock off by pressing theShift
key. The keyboard outputs pass or fail if following the test cases.Note: When testing in a virtual machine, use an on-screen keyboard (in VirtualBox: Input/Keyboard/Soft Keyboard) and observe the caps lock indicator of the on-screen keyboard. Using the hardware keyboard might show side effects with caps lock. Except for TEST_CAPSONLY-5 which can only be reliably tested on a hardware keyboard on host OS (not a VM). For windows 10 and windows 11 with a virtual box vm-onscreen keyboard, the following happens. The VM soft keyboard does NOT actually send the Shift Shift Key Stroke through but rather will change the keys pressed for example if an
a
is pressed the soft keyboard itself will change that key to aA
. This means we can't Test TEST_CAPONLY-5 on a soft keyboard.Prerequisites before each test
CapsOnOnly
andShiftFreesCaps
stores set, e.g.shift_frees_caps.kmp
.Test cases
click to expand
- **TEST_CAPSONLY-1**: no caps - press and release `1` Expected result: - output: `pass.` - **TEST_CAPSONLY-2**: caps - press and release `CapsLock` - press and release `2` Expected result: - caps lock indicator turned on - output: `pass.` - **TEST_CAPSONLY-3**: caps doesn't toggle - press and release `CapsLock` - press and release `CapsLock` - press and release `6` Expected result: - caps lock indicator turned on - output: `pass.` - **TEST_CAPSONLY-4**: shift turns off - press and release `CapsLock` - press and hold `Shift` - press and release `3` - release `Shift` Expected result: - caps lock indicator turned off - output: `pass.` - **TEST_CAPSONLY-5**: shift by itself turns off Be aware of limitations when testing this on virtual machines as noted above. - press and release `CapsLock` - press and release `Shift` Expected result: - caps lock indicator turned off - (no output)SUITE_TSF_APP:
Background: Some windows applications use the windows TSF these include: MS Word, Firefox, Fieldworks and the windows search field on the taskbar.
Setup Text Services Framework(TSF) testing with Keyman for Windows
Install Keyman Developer version 15 or greater.
Open the Keyman Developers Test page in Firefox. It must be
Firefox Browser
i. By default this is at
localhost:8008
so you could type this location in to the firefox address bar.ii. You can also access it by pressing the
Test package on web
button found in theCompile
tab for any keyboard package.On the
Keymand Developer Keyboard Test Page
use theKeyboard
dropdown to selectsystem keyboard
Use the Keyman for Windows icon located in the
System Tray
to select the desired Keyman for the Windows keyboard.Typing text into the input field will now use the TSF with the keyboard selected in step 4.
Test cases: double processing
click to expand
To run these tests the EuroLatin (SIL) Keyboard from [here](https://keyman.com/keyboards/sil_euro_latin). - GROUP_WIN10 - GROUP_WIN11 - **TEST_DOUBLE_PROCESSING_FIREFOX**: 1. Follow steps 1-4 from `Setup Text Services Framework(TSF) testing with Keyman for Window` above selecting the EuroLatin Keyboard at step 4. 2. Select the EuroLatin Keyboard 3. Type in to the windows search bar `a` `b` `c` `d` Backspace 4. Expected result is `abc` - **TEST_DOUBLE_PROCESSING_SEARCHBAR**: 1. Start Keyman 2. Select the EuroLatin Keyboard 3. Type in to the windows search bar `a` `b` `c` `d` Backspace 4. Expected result is `abc` ![Search_bar](https://user-images.githubusercontent.com/58423624/152917408-785a34be-ad1e-4e06-bcc6-bea2bd86d70d.png) - **TEST_DOUBLE_PROCESSING_NOTEPAD**: 1. Open Notepad 2. Start Keyman 2. Select the EuroLatin Keyboard 3. Type a sequence of letters `a` `b` `c` `d` 4. Expected result: `abcd` ---SUITE_IMX_KEYBOARDS
Test IMX keyboards still work after core integration. see tests suite on #5936 and also #6187
This is the keyboard to be tested. imsample.zip If an IMTest keyboard is already installed you will need to uninstall it first, restart windows and then install the Imsample keyboard attached to this PR.
I have copied the tests from #5936 and made a few modifications.
The overall test we are aiming for is that in doing the tests below the keyboard does not cause a keyman or app crash.
Test cases
click to expand
#### IMSAMPLE_KEYBOARD_INLINE_MENU: This keyboard uses the letters `aeom` to allow IMX input. This keyboard is a bit flakey being a demo and not polished. It also takes some setting up the following registry keys need to be added and set once installed. The location of the registry keys are at. `Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine\Active Keyboards\imsample` and are of type REG_DWORD The keys are `ShowIMWindow` and `ShowIMWindowAlways` they both need to be set to 0. In this mode IM window is not shown rather the app text is updated with a menu when either `aeom` is pressed. For example when type `he` the text will become `h[1ɛ 2ɜ 3ə 4e 5ɘ]` then you type the number `3` the text displayed should now be `hə` * **TEST_IMSAMPLE_INPUT:** - Install Imsample keyboard attached to this PR. - Select the keyboard its name is `IMTest` - Type some keys that are not the special keys e.g. `t` `r` `y` - - Type the `space` key then `a` then type 1 to get the 1st option - Expected Result: `try æ` - continue to the next test * **TEST_IMSAMPLE_INPUT_CONT:** ... contd from above - Type 'm' and type 3 to get `###` option - Expected Result: `try æ###` * **TEST_IMSAMPLE_BACKSPACE:** Using Notepad or Libreoffice - Type `f` - Type `e` - The screen should have f[1ɛ 2ɜ 3ə 4e 5ɘ] - Now press Backspace. the whole option menu including the square brackets `[` `]` should be deleted. - Expected result `f` #### IMSAMPLE_KEYBOARD_IM_WINDOW: This keyboard uses the letters `aeom` to allow IMX input. This time a IM window should display. ![image](https://user-images.githubusercontent.com/58423624/157192172-2ba2d0e4-8917-49b3-9709-00d7e9014645.png) The first time you use the keyboard with an application the registry keys will be created and by default set to 1. If the IM window is not appearing you will need to verify the value of two registry keys. The location of the registry keys are at. `Computer\HKEY_CURRENT_USER\SOFTWARE\Keyman\Keyman Engine\Active Keyboards\imsample` and are of type REG_DWORD The keys are `ShowIMWindow` and `ShowIMWindowAlways` they both need to be set to 1. Note for the IM window you need to use the mouse pointer to select the options, you can't type the number. * **TEST_IMSAMPLE_INPUT_IMW:** - Install Imsample keyboard attached to this PR (if not done already) - Select the keyboard its name is `IMTest` - Type some keys that are not the special keys e.g. `t` `r` `y` - Type the `space` key then `a` then click the 1st option - Expected Result: `try æ` - continue to the next test * **TEST_IMSAMPLE_INPUT_IMW_CONT:** ... contd from above - Type 'm' and select the 3rd `###` option - Expected Result: `try æ###` #### SIMPLIFIED_CHINESE: This keyboard will display the IMX window as soon as a string the matches the pinyin for one or more characters are typed. For all these tests install the Simplified Chinese Keyboard cs-pinyin.cmp found [here](https://keyman.com/keyboards/cs-pinyin) * **TEST_SIMPLIFIED_CHINESE_SINGLE**: Using Notepad or equivalent. - Type `o` - The IMX window should appear - Choose the 4th option - Expected result: 喔 * **TEST_SIMPLIFIED_CHINESE_MULTIPLE**: Using Notepad or equivalent. - Type `h` `a` `n` `z` `i` - The IMX window will appear and in the top left the letters `hanzi` will be present - Choose the 5th option - Expected result: 汉字变换 * **TEST_SIMPLIFIED_CHINESE_BACKSPACE_1**: Using Notepad or equivalent. This is to test the backspace occurring mid pinyin sequence before a character is output to the app - Type `h` `a` `n` `z` `i` - The IMX window will appear and in the top left the letters `hanzi` will be present - Press Backspace twice - The IMX window should now have the top left letters of `han`. - Choose the 5th option - Expected result: 汗 * **TEST_SIMPLIFIED_CHINESE_BACKSPACE_2**: Using Notepad or equivalent. This is to test the backspace of already output characters. - Type `h` `a` `n` `z` `i` - The IMX window will appear and in the top left the letters `hanzi` will be present. - Choose the 5th option - Expected result: 汉字变换 - Press Backspace twice - Expected result: 汉字SUITE_STORE_AND_CONTEXT
Test the options keyboard stores and context updating as a result of keystroke input. Introduced after #7077 and #7809 there are more tests on those PRs.
Test cases
click to expand
The key referred to as "enter" is alternatively labelled "return" on some keyboards. Keyboards for the following test are found in the zip file [store_context_kbd.zip](https://github.com/keymanapp/keyman/files/9441852/store_context_kbd.zip) **TEST_21_OPTION_STORE:** 1. Install **021 - options.kmx** 2. Open Notepad 3. Select 021-options.kmx as the current keyboard. 4. Type a1a0a Expected Result: "no foo.foo.no foo" **TEST_OUTPUT_KEYSTROKE** 1. Install **043 - output and keystroke.kmx** 2. Open Notepad 3. Select 043 - output and keystroke.kmx as the current keyboard. 4. Type 123 Expected Result: "abd3" **TEST_OUTPUT_KEYSTROKE_INVALID_CONTEXT** 1. If not installed from the previous test - Install **043 - output and keystroke.kmx** 2. Open Notepad 3. Select 043 - output and keystroke.kmx as the current keyboard. 4. Type 12Enter3 Expected Result: \ "abc\ 3" **TEST_DEADKEY_AND_CONTEXT** 1. Install **045 - deadkey and context.kmx** 2. Open Notepad 3. Select 045 - deadkey and context.kmx as the current keyboard. 4. Type yzShift+/ Expected Result: "correct"