Closed MakaraSok closed 1 year ago
Test specification and instructions
TEST_BASELINE_ADD_SIL_IPA (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KM (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_SPANISH (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_SWEDISH (FAILED): Tested this as per the instructions in web/samples/minified.html page and it is showing an error message.
TEST_ATTACHMENT_API (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working both auto and manual attachment modes.
TEST_NORMAL_USE (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_ELEMENT_HOPPING (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_SPECIFIC_KEYBOARDS (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_JAPANESE_TYPING (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_JAPANESE_FOCUS (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KM (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_SPANISH (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_SWEDISH (FAILED): Tested this as per the instructions in web/samples/minified.html page and it is showing an error message.
TEST_ATTACHMENT_API (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working both auto and manual attachment modes.
TEST_NORMAL_USE (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_ELEMENT_HOPPING (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_SPECIFIC_KEYBOARDS (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_JAPANESE_TYPING (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_JAPANESE_FOCUS (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KM (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_SPANISH (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_SWEDISH (FAILED): Tested this as per the instructions in web/samples/minified.html page and it is showing an error message.
TEST_ATTACHMENT_API (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working both auto and manual attachment modes.
TEST_NORMAL_USE (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_ELEMENT_HOPPING (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_SPECIFIC_KEYBOARDS (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_JAPANESE_TYPING (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_JAPANESE_FOCUS (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_BASELINE_ADD_SIL_IPA (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KM (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_KHMER_ANGKOR (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_ADD_SPANISH (PASSED): Tested this as per the instructions in web/samples/minified.html page and it is working as expected.
TEST_BASELINE_SWEDISH (FAILED): Tested this as per the instructions in web/samples/minified.html page and it is showing an error message.
TEST_ATTACHMENT_API (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working both auto and manual attachment modes.
TEST_NORMAL_USE (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_ELEMENT_HOPPING (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_SPECIFIC_KEYBOARDS (PASSED): Tested this as per the instructions in web/testing/attachment-api page and it is working as expected.
TEST_JAPANESE_TYPING (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
TEST_JAPANESE_FOCUS (PASSED): Tested this as per the instructions in web/samples/unminified.html page and it is working as expected.
The Swedish test was failing because we had a deployment issue with version 2.0 of sil_euro_latin, which I have since resolved on the backend.
@keymanapp-test-bot retest SUITE_DESKTOP GROUP_WINDOWS_FIREFOX TEST_BASELINE_SWEDISH GROUP_WINDOWS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_SAFARI TEST_BASELINE_SWEDISH GROUP_MACOS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_FIREFOX TEST_BASELINE_SWEDISH
@keymanapp-test-bot retest SUITE_DESKTOP GROUP_WINDOWS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_SAFARI TEST_BASELINE_SWEDISH GROUP_MACOS_CHROME TEST_BASELINE_SWEDISH
@bharanidharanj I cannot replicate the behavior. Let's retest these in a rubber ducky session.
@jahorton @mcdurdin Makara and I have retested the Swedish Keyboard issue in my Windows OS as well as in the Mac Mini M1 OS and we were able to reproduce it. But Makara was not able to reproduce this issue in his Virtual Machine. I am using Native Machine. (Windows and MacMini). So, we are confused whether the Native machine makes this kind of problem?
- TEST_BASELINE_SWEDISH (BLOCKED): Not able to retest this issue since it is throwing an error message after selecting the Swedish Keyboard.
I have flushed this file from the CDN cache to try and ensure there is nothing wrong there. Can you try clearing your browser cache and retrying these tests?
@keymanapp-test-bot retest SUITE_DESKTOP GROUP_WINDOWS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_SAFARI TEST_BASELINE_SWEDISH GROUP_MACOS_CHROME TEST_BASELINE_SWEDISH
@mcdurdin Okay, I will retest it as per your instructions.
Looks like everything in KMW worked well; sad there was some odd cache / s.keyman.com behavior that interfered at points, though.
The cache issue with s.keyman.com relates to failed deployments we have been experiencing and for which I have a patch in the pipeline
Keyman for Web Acceptance Test Procedures
Remember that KeymanWeb is embedded into the iOS and Android apps, thus we need to ensure that those work properly with a KeymanWeb release as well. See the Android and iOS instructions for details.
User Testing
SUITE_INSPECT: Visual Inspection of 16.0.29 Alpha
HISTORY.md
contains information on all the current changes.SUITE_DESKTOP: Desktop Platforms
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
Use the
web/samples/minified.html
sample page for the following tests.TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.ŋ
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ឞ
. Delete the character again.ស្រុក
.ស្រុ
ស្រ
ស្
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.Text Selection Tests [SKIP THIS FOR NOW AS THE KEYBOARD IS NOT SET UP FOR YET]
@mcdurdin Please at the following keyboard for this test. Thanks.
These tests are all run with the web_context_tests keyboard, in the Unminified Test Page. They should be run on both INPUT and TEXTAREA elements. Press Ctrl+F5 to reload the page between tests.
TEST_SELECTION: Type abcx. Select
bc
and type q. The output after the final key should beaqx
. (This is testing selection replacement with no matching keyboard rule.)TEST_SELECTION_2: Type abcx. Select
bc
and type p. The output after the final key should beaqx
. (This is testing selection replacement with a matching keyboard rule.)TEST_CONTEXT_BASE: Type abcd. The output after the final key should be
!
.TEST_CONTEXT_SELECTION: Type abcx. Select the
x
character, and type d. The output after the final key should beabcd
.TEST_CONTEXT_SELECTION_2: Type abcx. Select the
x
character, and delete it with Backspace. Type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_3: Type abcx. Select the
x
character, and type y. Press Backspace, and type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_4: Type xabcx. Select the
abc
characters, and type d. The output after the final key should bexdx
.Element Attachment Tests
web/testing/attachment-api
samples page to test the following:'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the
Tests the new Attachment/Enablement API functionality
test page.TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.Tests CJK keyboard functionality
Use the standard "Test unminified KeymanWeb" page.
TEST_JAPANESE_TYPING: Verify that a CJK keyboard still works.
japanese
, then click Add.a
. A "picker" displaying a few options should display:2
. The second option should replace the context.TEST_JAPANESE_FOCUS: Verify that a CJK keyboard is shown and hidden properly without interfering with a page's UX.
_TYPING
test).SUITE_TOUCH: Touch platform tests
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
Use the
web/samples/minified.html
sample page for the following tests.TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.ŋ
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ឞ
. Delete the character again.ស្រុក
.ស្រុក
.ស្រុ
ស្រ
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.TEST_BASELINE_CAMEROON: If using a touch-based keyboard, add the following keyboard by name: "sil_cameroon_qwerty"
Element Attachment Tests
web/testing/attachment-api
samples page to test the following:'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the
Tests the new Attachment/Enablement API functionality
test page.TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.