Closed MakaraSok closed 1 year ago
Test specification and instructions
š„ GROUP_ANDROID_5:
š„ GROUP_ANDROID_8:
ā GROUP_ANDROID_11:
History File for the Beta build 15.0.218:
Warning box appears after closing the first (Attention) dialog box:
Version number appears at the bottom of the page:
Install Keyboard (with Internet):
Install Keyboard (without Internet):
TEST_KEYMAN_SETTTINGS (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_SHARE (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_KEYMAN_BROWSER (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_TEXT_SIZE (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_CLEAR_TEXT (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_INFO (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_INSTALL_UPDATES (SKIPPED): Not able to see the Update option. Seems to be optional. Hence, I skipped.
TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_ADJUST_K_HEIGHT (PASSED): Tested this on Android emulator API 26 and it is working as expected.
TEST_SPACEBAR_CAPTION (FAILED): Tested this on Android emulator API 26 and it seems that changing any options does not work. Seems to be an issue.
TEST_GET_STARTED (BLOCKED): TTested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_INSTALL_KD (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_KEYMAN_SETTTINGS (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_SHARE (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_KEYMAN_BROWSER (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_TEXT_SIZE (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_CLEAR_TEXT (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_INFO (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_INSTALL_UPDATES (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_CHANGE_DISPLAY_LANG (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_ADJUST_K_HEIGHT (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_SPACEBAR_CAPTION (BLOCKED): Tested this in Android 5.0 API 21 emulator and it showing Chrome update (even though installed the latest chrome version) error message after installing the Keyman 15.0 Beta build.
TEST_INAPP_LATIN_KEYBOARD (PASSED): Tested this with the latest Keyman 15.0.221 beta build in Android 8 API 26 emulator and it is working as expected.
TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this with the latest Keyman 15.0.221 beta build in Android 8 API 26 emulator and it is working as expected.
TEST_SYSTEM_LATIN_KEYBOARD (PASSED): Tested this with the latest Keyman 15.0.221 beta build in Android 8 API 26 emulator and it is working as expected.
TEST_KEYBOARD_PICKER (PASSED): Tested this with the latest Keyman 15.0.221 beta build in Android 8 API 26 emulator and it is working as expected.
TEST_EXT_BLUETOOTH_KEYBOARD (SKIPPED): Skipped this test since I don't have Bluetooth Keyboard for testing.
TEST_EXT_AZERTY_KEYBOARD (SKIPPED): Skipped this test since I don't have AZERTY Keyboard for testing.
TEST_INAPP_LATIN_KEYBOARD (PASSED): Tested this on Android 11 API 30 emulator with the latest Keyman 15.0.221 Beta build and it is working as expected.
TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this on Android 11 API 30 emulator with the latest Keyman 15.0.221 Beta build and it is working as expected.
TEST_SYSTEM_LATIN_KEYBOARD (PASSED): Tested this on Android 11 API 30 emulator with the latest Keyman 15.0.221 Beta build and it is working as expected.
TEST_KEYBOARD_PICKER (PASSED): Tested this on Android 11 API 30 emulator with the latest Keyman 15.0.221 Beta build and it is working as expected.
TEST_EXT_BLUETOOTH_KEYBOARD (SKIPPED): Skipped this test since I don't have Bluetooth Keyboard for testing.
TEST_EXT_AZERTY_KEYBOARD (SKIPPED): Skipped this test since I don't have Bluetooth Keyboard for testing.
@keymanapp-test-bot retest SUITE_MENU_FUNCTIONALITY GROUP_ANDROID_5 all, SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_5 all, SUITE_KB_DISTRIBUTION GROUP_ANDROID_5 all
I don't see the notification on updating Chrome in Android 5 as a blocking issue for any of the tests. All tests could be done without any issue, couldn't they?
@keymanapp-test-bot retest SUITE_KB_DISTRIBUTION GROUP_ANDROID_8 all, SUITE_KB_DISTRIBUTION GROUP_ANDROID_11 all
I can install the .kmp file as usual on both Android versions. Can you double check them?
@bharanidharanj
Some devices are needed to perform these tests: (1) a bluetooth keyboard or a USB keyboard which can be attached to the USB-C port of the physical Android phone
@keymanapp-test-bot retest SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_5 TEST_KEYBOARD_PICKER TEST_EXT_BLUETOOTH_KEYBOARD TEST_EXT_AZERTY_KEYBOARD, SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_8 TEST_KEYBOARD_PICKER TEST_EXT_BLUETOOTH_KEYBOARD TEST_EXT_AZERTY_KEYBOARD, SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_11 TEST_KEYBOARD_PICKER TEST_EXT_BLUETOOTH_KEYBOARD TEST_EXT_AZERTY_KEYBOARD
@keymanapp-test-bot retest SUITE_KB_DISTRIBUTION GROUP_ANDROID_5 TEST_KMP_DISTRIBUTION, SUITE_KB_DISTRIBUTION GROUP_ANDROID_8 TEST_KMP_DISTRIBUTION, SUITE_KB_DISTRIBUTION GROUP_ANDROID_11 TEST_KMP_DISTRIBUTION
Note:
@keymanapp-test-bot retest SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_5 TEST_EXT_AZERTY_KEYBOARD, SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_8 TEST_EXT_AZERTY_KEYBOARD, SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_11 TEST_EXT_AZERTY_KEYBOARD
Thank you for testing! Test failures with external AZERTY hardware keyboard tracked in #6875.
Thank you for testing! Test failures with external AZERTY hardware keyboard tracked in #6875.
@mcdurdin Thank you!
Keyman for Android Regression Test Procedures
These test procedures are to be run each sprint, also before moving from alpha to beta, or beta to stable, and ideally before PRs are merged into stable branches.
Copy these checklists of tests into a new issue and title/tag as required.
Gather Assets for Testing
Setup Steps
User Testing
SUITE_INSPECT: Visual Inspection of master branch
/history.md
contains all the current changesSUITE_MENU_FUNCTIONALITY: Menu Functionality Tests
click to expand
## Various versions of Androids UI for these tests assume default English locale * GROUP_ANDROID_5 * GROUP_ANDROID_8 * GROUP_ANDROID_11 - **TEST_GET_STARTED:** This tests the "Get Started" menu from a fresh installationExpand to see how to do and verify this
1. When the app loads, verify the "Get Started" menu is visible 2. From the "Get Started" menu, touch "Add a keyboard for your language" 3. Verify "Install Keyboard or Dictionary" menu appears 4. From the "Install Keyboard or Dictionary" menu, click the back arrow to return to "Get Started" 5. From the "Get Started" menu, touch "Enable Keyman as system-wide keyboard" 6. Verify Android system menus appear for controlling virtual keyboard 7. Scroll down and enable Keyman as a system keyboard. 8. On the "Attention" dialog, read and hit "OK" to dismiss the dialogs. 9. Touch the back button to return to "Get Started" 10. Verify "Enable Keyman as system-wide keyboard" now has a ticked checkbox. 11. From the "Get Started" menu, touch "Set Keyman as default keyboard" 12. On the "Change keyboard" dialog, select Keyman 13. Verify on the "Get Started" menu that "Set Keyman as default keyboard" now has a ticked checkbox. 14. From the "Get Started" menu, touch "More info" 15. Verify the Info page appears with a version string at the bottom of the page 16. Touch the back button (arrow) to return to "Get Started" 17. From the "Get Started" menu, untick the last option 'Show "Get Started" on startup' 18. Close the "Get Started" menu and exit the app. 19. From the Android device, start Keyman for Android 20. Verify the "Get Started" menu does not appear.Expand to see how to do and verify this
1. On the device, enable internet connectivity 2. Touch "Install from keyman.com" 3. Verify a new keyboard can be downloaded and selected 4. On the device, disable internet connectivity 5. From "Install Keyboard or Dictionary", touch "Install from keyman.com" 6. Verify device displays a message "Cannot connect to Keyman server!" 7. On the device, re-enable internet connectivityExpand to see how to do and verify this
1. From "Settings" --> "Installed Languages" 2. Select "English" --> English Settings 3. Select keyboard settings menu 4. Verify Info appears with keyboard version, help link, and QR code 5. If keyboard update is available, click on version and verify update 6. If help link is available, click on link 7. Verify keyboard help documentation appears 8. Return to English Settings menu 9. Select Dictionary --> English dictionary (MTNT) 10. Verify the Dictionary info appears 11. Return to English Settings menu 12. Turn on/off 'Enable Predictions' and validate each 13. Turn on/off 'Enable Corrections' and validate each. For example, when Corrections are enabled and context "tooo" should still suggest "too" 14. Validate that when both predictions and corrections are off, banner is not visible 15. Add/Remove dictionaries - validate 16. If multiple dictionaries are available, test swapping between themExpand to see how to do and verify this
1. On default page, click on text field and set system keyboard to Keyman 2. Verify can type with Keyman as system keyboard 3. Close and reopen Keyman appExpand to see how to do and verify this
1. From the device, disable internet access 2. From the Keyman app menu, select "Info" 3. Verify Keyman for Android version appears at the bottom of the screen 4. Verify Keyman for Android help documentation is displayed. 5. From the device, enable internet accessExpand to see how to do and verify this
1. Click on "Install Updates" 2. Accept "Download" on the dialogue 3. Verify resource is updated after background download completeExpand to see how to do and verify this
1. From "Settings" --> "Display Language" 2. Select "Khmer" 3. From "Settings" --> Verify menu selections in Khmer 4. From the overflow menu --> clear text 5. Verify Keyman text field has Khmer prompt (for "Start typing here") 6. From "Settings" --> "Change Display Language" 7. If the string is translated, the first row should be "Default Locale" in Khmer 8. Select "English" 9. Verify the app reloads in EnglishExpand to see how to do and verify this
1. Have the device in portrait orientation 2. From "Settings" --> "Adjust keyboard height" 3. On the menu, drag the OSK height to change the keyboard height 4. Click back 5. Verify the OSK is refreshed to the selected height 6. From "Settings" --> "Adjust keyboard height" 7. Click the "reset to defaults" button and then click back 8. Verify the OSK reverts to the original height 9. Rotate the device to landscape orientation 10. From "Settings" --> "Adjust keyboard height" 11. On the menu, drag the OSK height to change the keyboard height 12. Click back 13. Verify the OSK is refreshed to the selected height 14. From "Settings" --> "Adjust keyboard height" 15. Click the "reset to defaults" button and then click back 16. Verify the OSK reverts to the original height 17. Rotate the device back to portrait orientationExpand to see how to do and verify this
1. Select the default SIL EuroLatin keyboard 2. From "Settings" --> Spacebar caption 3. On the menu, select "Language" 4. Click back until the OSK is displayed 5. Verify the spacebar label is "English" 6. From "Settings" --> Spacebar caption 7. On the menu, select "Keyboard" 8. Click back until the OSK is displayed 9. Verify the spacebar label is "EuroLatin (SIL)" 10. From "Settings" --> Spacebar caption 11. On the menu, select "Blank" 12. Click back until the OSK is displayed 13. Verify the spacebar label is blank 14. From "Settings" --> Spacebar caption 15. On the menu, select "Language+Keyboard" 16. Click back until the OSK is displayed 17. Verify the spacebar label is "English - EuroLatin (SIL)"SUITE_KEYBOARD_FUNCTIONALITY: Keyboard Tests
click to expand
## Various versions of Androids UI for these tests assume default English locale * GROUP_ANDROID_5 * GROUP_ANDROID_8 * GROUP_ANDROID_11 ### In-app Keyboards These tests are run in the context of typing in the Keyman app - **TEST_INAPP_LATIN_KEYBOARD:** English(EuroLatin SIL) In portrait orientation, verify OSK is visible and fills the width the bottom of the screenExpand to see how to do and verify this
1. Verify long-press `q` key works 2. Verify long-press `p` key works 3. Verify uppercase layer can be selected via `SHIFT` 4. Verify number layer can be selected via `123` 5. Verify long-press `1` key works 6. Verify long-press `0` key works 7. Verify backspace, space, and enter keys work 8. Verify *Khmer* -> *Khmer Angkor* keyboard can be added via Settings menu 9. Repeat in landscape orientationExpand to see how to do and verify this
1. In the app, add/select a random non-Latin script keyboard 2. Verify OSK is visible 3. Test several keys and modifiers and verify Latin characters are not appearing.Expand to see how to do and verify this
1. Verify long-press `q` key works 2. Verify long-press `p` key works 3. Verify uppercase layer can be selected via `SHIFT` 4. Verify number layer can be selected via `123` 5. Verify long-press `1` key works 6. Verify long-press `0` key works 7. Verify backspace, space, and enter keys work 8. Verify can change to random non-Latin script keyboard via globe button 9. Repeat in landscape orientationExpand to see how to do and verify this
1. With only 1 Keyman keyboard installed and selected as a system keyboard, long-press and release the the globe button 2. Verify the bottom of the Keyboards picker menu lists other available input methods 3. Click on one of the other input methods 4. Verify the Keyman system keyboard switches to the selected input method 5. On the device's Settings, select Keyman as the input method 6. On the Keyman keyboard, short-press and release the globe button 7. Verify the previous input method is selectedExpand to see how to do and verify this
1. Start with English(EuroLatin SIL) as the selected keyboard 2. On the device's Settings, show the input method (OSK) 3. Verify can type with numbers, letters, and punctuation 4. Verify `caps lock` has no effect when typing 5. Verify long-press on a character repeats that character 6. Verify `tab` toggles to next field 7. Verify `enter` adds a newline 8. Verify `backspace` removes a character 9. Verify ctrl+tab brings up the keyboard picker menu 10. Use tab or arrow keys to navigate the keyboard picker menu 11. Select a non-Latin keyboard 12. Resume typing and verify new script appearsExpand to see how to do and verify this
1. Start Keyman and install the French Basic keyboard 2. From the physical keyboard, type on the top letter row (azerty) 3. Verify `azerty` appears 4. Type on the 102nd key 5. Verify `<` appears 6. Type shift-102nd key 7. Verify `>` appearsSUITE_KB_DISTRIBUTION: Ad-hoc Keyboard Distribution Tests
Expand to see how to do and verify this
## Various versions of Androids UI for these tests assume default English locale * GROUP_ANDROID_5 * GROUP_ANDROID_8 * GROUP_ANDROID_11 - **TEST_KMP_DISTRIBUTION** KMP distributionExpand to see how to do and verify this
1. Reset app settings (in Android Pie: Settings>Storage>Other apps>Keyman>Clear Storage) 2. Verify kmp [distribution](https://help.keyman.com/developer/current-version/guides/distribute/) successfully installsExample: [Khmer Angkor](https://downloads.keyman.com/keyboards/khmer_angkor/1.0.2/khmer_angkor.kmp) 3. Verify kmp successfully installs from Settings --> Install Keyboard or Dictionary --> Install from local file