Closed MakaraSok closed 2 years ago
Test specification and instructions
Early notes:
_HARDWARE
suffix about typing with the hardware keyboard on corresponding setups, but we need to make that more explicit. @jahorton Thanks for the good feedback on the test setup. We'll need to make sure that these notes are captured and applied to the regression test template for future tests for web as well.
@jahorton Thanks a lot for the prompt response. 💯
CJK keyboard functionality tests
from the current set...Android Version History:
..iOS Version History:
@bharanidharanj I note that for TEST_BASELINE_ADD_KHMER_ANGKOR you entered khmer_angkor
into the 'language name' field instead of the 'keyboard name' field. Can you retest, making sure you enter khmer_angkor
into the 'keyboard name' field?
@keymanapp-test-bot retest TEST_BASELINE_ADD_KHMER_ANGKOR
I note also that the Android version history is okay -- we just have no changes in the 16.0 branch for Android as yet.
@bharanidharanj I note that for TEST_BASELINE_ADD_KHMER_ANGKOR you entered
khmer_angkor
into the 'language name' field instead of the 'keyboard name' field. Can you retest, making sure you enterkhmer_angkor
into the 'keyboard name' field?@keymanapp-test-bot retest TEST_BASELINE_ADD_KHMER_ANGKOR
@mcdurdin Okay, Marc. I will retest it. Thanks.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_BASELINE_ADD_KM (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_BASELINE_ADD_SPANISH (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_BASELINE_SWEDISH (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_BASELINE_CAMEROON (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_ATTACHMENT_API (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_NORMAL_USE (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_ELEMENT_HOPPING (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
TEST_SPECIFIC_KEYBOARDS (PASSED): Tested this as per the instructions in my Android Mobile (Ver 11) and it is working as expected.
Keyman for Web Acceptance Test Procedures
Remember that KeymanWeb is embedded into the iOS and Android apps, thus we need to ensure that those work properly with a KeymanWeb release as well. See the Android and iOS instructions for details.
User Testing
SUITE_INSPECT: Visual Inspection of alpha-test-6525 branch
HISTORY.md
contains information on all the current changes.SUITE_DESKTOP: Desktop Platforms
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
Use the
web/samples/minified.html
sample page for the following tests.TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.ŋ
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ឞ
. Delete the character again.ស្រុក
.ស្រុ
ស្រ
ស្
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.Text Selection Tests
These tests are all run with the web_context_tests keyboard, in the Unminified Test Page. They should be run on both INPUT and TEXTAREA elements. Press Ctrl+F5 to reload the page between tests.
TEST_SELECTION: Type abcx. Select
bc
and type q. The output after the final key should beaqx
. (This is testing selection replacement with no matching keyboard rule.)TEST_SELECTION_2: Type abcx. Select
bc
and type p. The output after the final key should beaqx
. (This is testing selection replacement with a matching keyboard rule.)TEST_CONTEXT_BASE: Type abcd. The output after the final key should be
!
.TEST_CONTEXT_SELECTION: Type abcx. Select the
x
character, and type d. The output after the final key should beabcd
.TEST_CONTEXT_SELECTION_2: Type abcx. Select the
x
character, and delete it with Backspace. Type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_3: Type abcx. Select the
x
character, and type y. Press Backspace, and type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_4: Type xabcx. Select the
abc
characters, and type d. The output after the final key should bexdx
.Element Attachment Tests
web/testing/attachment-api
samples page to test the following:'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the
Tests the new Attachment/Enablement API functionality
test page.TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.Tests CJK keyboard functionality
Use the standard "Test unminified KeymanWeb" page.
TEST_JAPANESE_TYPING: Verify that a CJK keyboard still works.
japanese
, then click Add.a
. A "picker" displaying a few options should display:2
. The second option should replace the context.TEST_JAPANESE_FOCUS: Verify that a CJK keyboard is shown and hidden properly without interfering with a page's UX.
_TYPING
test).SUITE_TOUCH: Touch platform tests
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
Use the
web/samples/minified.html
sample page for the following tests.TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.ŋ
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ឞ
. Delete the character again.ស្រុក
.ស្រុក
.ស្រុ
ស្រ
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.TEST_BASELINE_CAMEROON: If using a touch-based keyboard, add the following keyboard by name: "sil_cameroon_qwerty"
Element Attachment Tests
web/testing/attachment-api
samples page to test the following:'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the
Tests the new Attachment/Enablement API functionality
test page.TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.