Closed jahorton closed 10 months ago
Test specification and instructions
✅ GROUP_ANDROID_CHROME_TOUCH:
✅ GROUP_ANDROID_CHROME_HARDWARE:
✅ GROUP_IPHONE_SAFARI_TOUCH:
✅ GROUP_IPHONE_SAFARI_HARDWARE:
🟥 GROUP_IPAD_SAFARI_TOUCH:
🟥 GROUP_IPAD_SAFARI_HARDWARE:
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that the key sequence n> produced Å‹.
TEST_BASELINE_ADD_KM (PASSED): Verified that the Khmer_Basic keyboard was added.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that the Khmer_angkor keyboard was added. Able to see ស្រុក by typing key sequence s u j r k. Able to see that the reorder rule test, by hitting Backspace key.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that the Spanish keyboard was added to the list.
TEST_BASELINE_SWEDISH (PASSED): Verified that the given key sequences shows the expected output.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that the key sequence n> produced Å‹.
TEST_BASELINE_ADD_KM (PASSED): Verified that the Khmer_Basic keyboard was added.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that the Khmer_angkor keyboard was added. Able to see ស្រុក by typing key sequence s u j r k. Able to see that the reorder rule test, by hitting Backspace key.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that the Spanish keyboard was added to the list.
TEST_BASELINE_SWEDISH (PASSED): Verified that the given key sequences shows the expected output.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that the key sequence n> produced Å‹.
TEST_BASELINE_ADD_KM (PASSED): Verified that the Khmer_Basic keyboard was added.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that the Khmer_angkor keyboard was added. Able to see ស្រុក by typing key sequence s u j r k. Able to see that the reorder rule test, by hitting Backspace key.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that the Spanish keyboard was added to the list.
TEST_BASELINE_SWEDISH (PASSED): Verified that the given key sequences shows the expected output.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that the key sequence n> produced Å‹.
TEST_BASELINE_ADD_KM (PASSED): Verified that the Khmer_Basic keyboard was added.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that the Khmer_angkor keyboard was added. Able to see ស្រុក by typing key sequence s u j r k. Able to see that the reorder rule test, by hitting Backspace key.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that the Spanish keyboard was added to the list.
TEST_BASELINE_SWEDISH (PASSED): Verified that the given key sequences shows the expected output.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that the key sequence n> produced Å‹.
TEST_BASELINE_ADD_KM (PASSED): Verified that the Khmer_Basic keyboard was added.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that the Khmer_angkor keyboard was added. Able to see ស្រុក by typing key sequence s u j r k. Able to see that the reorder rule test, by hitting Backspace key.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that the Spanish keyboard was added to the list.
TEST_BASELINE_SWEDISH (PASSED): Verified that the given key sequences shows the expected output.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that the key sequence n> produced Å‹.
TEST_BASELINE_ADD_KM (PASSED): Verified that the Khmer_Basic keyboard was added.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that the Khmer_angkor keyboard was added. Able to see ស្រុក by typing key sequence s u j r k. Able to see that the reorder rule test, by hitting Backspace key.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that the Spanish keyboard was added to the list.
TEST_BASELINE_SWEDISH (PASSED): Verified that the given key sequences shows the expected output.
TEST_SELECTION (PASSED): Tested in unminified Keymanweb test page and tested in both INPUT and TEXTAREA elements and here is my observations: 1. Verified that the output shows aqx.
TEST_SELECTION_2 (PASSED): Verified that the output after the final key shows aqx.
TEST_CONTEXT_BASE (PASSED): Verified that the output finally shows !.
TEST_CONTEXT_SELECTION (PASSED): Verified that the output shows abcd.
TEST_CONTEXT_SELECTION_2 (PASSED): Verified that the final key shows !.
TEST_CONTEXT_SELECTION_3 (PASSED): Verified that the final key shows !.
TEST_CONTEXT_SELECTION_4 (PASSED): Verified that the final key shows xdx.
TEST_SELECTION_SHIFT (PASSED): Verified that the pressing shift shows abcx with bc still selected.
..inputfield
TEST_ATTACHMENT_API (PASSED): 1. Verified that the 'auto' attachment mode properly and automatically attached to each dynamically-added control. 2. Verified the 'manual' attachment mode results in a page with no KMW-activated controls. 3. Verified that each 'attached' control functions as expected for the selected keyboard(s). 4. Verified that the KeymanWeb properly attaches to and detaches from each relevant type of control. Also, verified that the independent-keyboard management API functions correctly.
TEST_NORMAL_USE (PASSED): Tested in the given test page and here is my observation : 1. Verified that the keyboard displays and works correctly when the first loaded up.
TEST_ELEMENT_HOPPING (PASSED): Verified that the OSK remains visible if I jump to another page element and it is automatically hidden if I clicked on the blank area.
TEST_SPECIFIC_KEYBOARDS (PASSED): Verified that the keyboard changes from one dynamic area to other and it is showing the expected keyboard layout. Seems to be working fine.
TEST_JAPANESE_TYPING (PASSED): 1. Able to add japanese keyboard. 2. Able to select the second option to replace the existing context.
TEST_JAPANESE_FOCUS (PASSED): 1. Verified that it is possible to see the japanese keyboard when I clicked either in text area or the input field. 2. If I clicked in the black area on the page, the keyboard disappeared. Seems to be working fine.
@bharanidharanj
- TEST_TYPING_AUTOSCROLL (FAILED): Verified that the text autoscroll is not working in INPUT field.
..inputfield
... but it did autoscroll. Note the left-hand side of the input field. I'm assuming that you just typed the word testing
over and over again... so why is the first thing visible in the input field ting
, not testing
? It's because the first copy's tes
got "autoscrolled" off the left-hand side.
Note how you can still see the caret at the end of the input field, too:
From the test's definition:
Verify that the text is automatically scrolled so that the last three characters and the caret (the thin, blinking | mark for text position) are visible.
Your ing
is visible, as well as the caret. It totally PASSES.
@bharanidharanj
- TEST_TYPING_AUTOSCROLL (FAILED): Verified that the text autoscroll is not working in INPUT field.
..inputfield
... but it did autoscroll. Note the left-hand side of the input field. I'm assuming that you just typed the word
testing
over and over again... so why is the first thing visible in the input fieldting
, nottesting
? It's because the first copy'stes
got "autoscrolled" off the left-hand side.Note how you can still see the caret at the end of the input field, too:
From the test's definition:
Verify that the text is automatically scrolled so that the last three characters and the caret (the thin, blinking | mark for text position) are visible.
Your
ing
is visible, as well as the caret. It totally PASSES.
@jahorton I thought after typing the last 3 characters the word 'testing' will jump into the next line (ie., move down to next line ..like in text area). Now, I can understand the procedure. Thanks for the Clarification. :-)
@keymanapp-test-bot retest SUITE_DESKTOP TEST_TYPING_AUTOSCROLL (To reset the test that just got clarified)
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that pressing the subkey of n - shows the expected output.
TEST_BASELINE_ADD_KM PASSED): Verified that able to see the Khmer_basic keyboard.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that typing the given sequence of letters on touch produces the expected output. Also, verified that pressing the Backspace key, delete the character one by one as per the reorder rule.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that a Spanish keyboard is added to the list.
TEST_BASELINE_SWEDISH (PASSED): 1. Verified that the two characters does not combine. 2. Verified that long pressing p or long pressing G displays the subkey menu and clicking on that produces expected output.
TEST_BASELINE_CAMEROON (PASSED): 1. Verified that the top two rows of the symbol layer working correctly. 2. Verified that the tri-color key changes layer appropriately and that keys pressed automatically transition back to the default layer. 3. Verified that the subkey on the 'default' and on the 'shift' layers output the expected character.
TEST_BASELINE_AUTOSCROLL PASSED): 1. Verified that the text is automatically scrolled so that the last three characters and the caret are fully visible. 2. Verified that the Keyboard is still visible at any point while typing in the input field.
TEST_ROTATION (PASSED): 1. Verified that the OSK repositions itself at the bottom of the screen and occupies the full width of the device. (in both the roations - horizontal and vertical mode)
TEST_ATTACHMENT_API (PASSED): 1. Verified that the 'auto' attachment mode properly and automatically attached to each dynamically-added control. 2. Verified the 'manual' attachment mode results in a page with no KMW-activated controls. 3. Verified that each 'attached' control functions as expected for the selected keyboard(s). 4. Verified that the KeymanWeb properly attaches to and detaches from each relevant type of control. Also, verified that the independent-keyboard management API functions correctly.
TEST_NORMAL_USE (PASSED): Tested in the given test page and here is my observation : 1. Verified that the keyboard displays and works correctly when the first loaded up.
TEST_ELEMENT_HOPPING (PASSED): Verified that the OSK remains visible if I jump to another page element and it is automatically hidden if I clicked on the blank area.
TEST_SPECIFIC_KEYBOARDS (PASSED): Verified that the keyboard changes from one dynamic area to other and it is showing the expected keyboard layout. Seems to be working fine.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Verified that pressing the subkey of n - shows the expected output.
TEST_BASELINE_ADD_KM (PASSED): Verified that able to see the Khmer_basic keyboard.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Verified that typing the given sequence of letters on touch produces the expected output. Also, verified that pressing the Backspace key, delete the character one by one as per the reorder rule.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that a Spanish keyboard is added to the list.
TEST_BASELINE_SWEDISH (PASSED): 1. Verified that the two characters does not combine. 2. Verified that long pressing p or long pressing G displays the subkey menu and clicking on that produces expected output.
TEST_BASELINE_CAMEROON (PASSED): 1. Verified that the top two rows of the symbol layer working correctly. 2. Verified that the tri-color key changes layer appropriately and that keys pressed automatically transition back to the default layer. 3. Verified that the subkey on the 'default' and on the 'shift' layers output the expected character.
TEST_BASELINE_AUTOSCROLL (PASSED): 1. Verified that the text is automatically scrolled so that the last three characters and the caret are fully visible. 2. Verified that the Keyboard is still visible at any point while typing in the input field.
TEST_ROTATION (PASSED): 1. Verified that the OSK repositions itself at the bottom of the screen and occupies the full width of the device. (in both the rotations - horizontal and vertical mode)
TEST_ATTACHMENT_API (PASSED): 1. Verified that the 'auto' attachment mode properly and automatically attached to each dynamically-added control. 2. Verified the 'manual' attachment mode results in a page with no KMW-activated controls. 3. Verified that each 'attached' control functions as expected for the selected keyboard(s). 4. Verified that the KeymanWeb properly attaches to and detaches from each relevant type of control. Also, verified that the independent-keyboard management API functions correctly.
TEST_NORMAL_USE (PASSED): Tested in the given test page and here is my observation : 1. Verified that the keyboard displays and works correctly when the first loaded up.
TEST_ELEMENT_HOPPING (PASSED): Verified that the OSK remains visible if I jump to another page element and it is automatically hidden if I clicked on the blank area.
TEST_SPECIFIC_KEYBOARDS (PASSED): Verified that the keyboard changes from one dynamic area to other and it is showing the expected keyboard layout. Seems to be working fine.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Tested in iOS 16.4 / iPad (10th generation) Simulator and here is my observation: Verified that pressing the subkey of n - shows the expected output.
TEST_BASELINE_ADD_KM (PASSED): Verified that able to see the Khmer_basic keyboard.
TEST_BASELINE_ADD_KHMER_ANGKOR (FAILED): Khmer Angkor keyboard does not show up the subkey.
TEST_BASELINE_ADD_SPANISH (PASSED): Verified that a Spanish keyboard is added to the list.
TEST_BASELINE_SWEDISH (PASSED): 1. Verified that the two characters does not combine. 2. Verified that long pressing p or long pressing G displays the subkey menu and clicking on that produces expected output.
TEST_BASELINE_CAMEROON (PASSED): 1. Verified that the top two rows of the symbol layer working correctly. 2. Verified that the tri-color key changes layer appropriately and that keys pressed automatically transition back to the default layer. 3. Verified that the subkey on the 'default' and on the 'shift' layers output the expected character.
TEST_BASELINE_AUTOSCROLL (PASSED): 1. Verified that the text is automatically scrolled so that the last three characters and the caret are fully visible. 2. Verified that the Keyboard is still visible at any point while typing in the input field.
TEST_ROTATION (PASSED): 1. Verified that the OSK repositions itself at the bottom of the screen and occupies the full width of the device. (in both the roations - horizontal and vertical mode) ..Portrait
..Landscape
TEST_SPECIFIC_KEYBOARDS (PASSED): Verified that the keyboard changes from one dynamic area to other and it is showing the expected keyboard layout. Seems to be working fine.
OK... so the two test failures are because the tablet form-factor layout for khmer_angkor
doesn't use subkeys to the same extent as the phone form-factor layout. I've gone ahead and adjusted the description accordingly.
Looks like everything else passed swimmingly, so... I think that means we're good to go.
For updating the Platform Acceptance Test for Web wiki page:
SUITE_INSPECT
in its original position.Keymanweb_TestPullRequests/390024:id/src
=> Keymanweb_TestPullRequests/TBD:id/src
, since that's the pattern we've been using for the platform-acceptance user testing template.As this was a testing issue for that branch, we can retire this issue now.
Keyman for Web Acceptance Test Procedures
For this test run, please refer to the builds posted on https://github.com/keymanapp/keyman/pull/8560#issuecomment-1492750092.
As the feature branch has relocated test page resources, note that the links contained herein are updated to point to their new locations. A few extra tests have also been added.
Once the feature-branch lands, updating the regression test spec to use the links from this one will resolve #9023 moving forward. Some of the links may not work on
master
, though, hence my hesitancy to update them at the moment.I have dropped SUITE_INSPECT for this run, as it's not against an official release of Keyman, but the rest of these tests should otherwise match what is seen at https://github.com/keymanapp/keyman/wiki/Platform-Acceptance-Test-for-Web at this time.
User Testing
SUITE_DESKTOP: Desktop Platforms
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
From the testing index page, use the first link ("View Keymanweb use samples") and then select "Example 1" for the following tests.
TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.Å‹
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ážž
. Delete the character again.ស្រុក
.ស្រុ
ស្រ
ស្
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.Text Selection Tests
These tests are all run with the web_context_tests keyboard on the Test unminified Keymanweb test page. They should be run on both INPUT and TEXTAREA elements. Press Ctrl+F5 to reload the page between tests.
TEST_SELECTION: Type abcx. Select
bc
and type q. The output after the final key should beaqx
. (This is testing selection replacement with no matching keyboard rule.)TEST_SELECTION_2: Type abcx. Select
bc
and type p. The output after the final key should beaqx
. (This is testing selection replacement with a matching keyboard rule.)TEST_CONTEXT_BASE: Type abcd. The output after the final key should be
!
.TEST_CONTEXT_SELECTION: Type abcx. Select the
x
character, and type d. The output after the final key should beabcd
.TEST_CONTEXT_SELECTION_2: Type abcx. Select the
x
character, and delete it with Backspace. Type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_3: Type abcx. Select the
x
character, and type y. Press Backspace, and type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_4: Type xabcx. Select the
abc
characters, and type d. The output after the final key should bexdx
.TEST_SELECTION_SHIFT: (new) Type abcx. Select
bc
and then press the left Shift key on the OSK. The output after pressing shift should beabcx
, withbc
still selected.TEST_TYPING_AUTOSCROLL: (new) Type a very long string of text into the "input field" element using the "English - English" keyboard until the text reaches its right-hand side.
|
mark for text position) are visible.Element Attachment Tests
'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the Attachment/Enablement API functionality test page.
TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.Tests CJK keyboard functionality
Use the standard Test unminified Keymanweb test page.
TEST_JAPANESE_TYPING: Verify that a CJK keyboard still works.
japanese
, then click Add.a
. A "picker" displaying a few options should display:2
. The second option should replace the context.TEST_JAPANESE_FOCUS: Verify that a CJK keyboard is shown and hidden properly without interfering with a page's UX.
_TYPING
test).SUITE_TOUCH: Touch platform tests
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
From the testing index page, use the first link ("View Keymanweb use samples") and then select "Example 1" for the following tests.
TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.Å‹
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ážž
. Delete the character again.ស្រុក
.ស្រុក
.ស្រុ
ស្រ
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.TEST_BASELINE_CAMEROON: If using a touch-based keyboard, add the following keyboard by name: "sil_cameroon_qwerty"
TEST_BASELINE_AUTOSCROLL: (new) Type a very long string of text into the "input field" element using the "English - English" keyboard until the text reaches its right-hand side.
|
mark for text position) are visible.TEST_ROTATION: (new) Put the test device into 'portrait' orientation - with the longer dimension held vertically.
Element Attachment Tests
'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the Attachment/Enablement API functionality test page.
TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.