search

keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
372 stars 102 forks source link

fix(web): modular app/webview KMW hosted within Android app šŸ§© #8544

Closed jahorton closed 1 year ago

jahorton commented 1 year ago

šŸŽ‰ Big milestone! šŸŽ‰

With these changes in place, the Android app has been (lightly) tested to work with a fully-modular, bundled Keyman Engine for Web core!

Turns out there were a few loose threads here and there to mend and reconnect - it did take most of a day to debug the biggest issues and resolve things to the current state shown in this PR. But...

Other details:

šŸš§ Known issues šŸš§:

User Testing

Since there has been a lot of changes - some small, some large - under the hood in pursuit of full modularization, we're going to want a near-full regression-test suite on this PR. At minimum. I may add other tests at a later point. But that should give us a good testing baseline to start with.

Gather Assets for Testing

Setup Steps

  1. Uninstall previous version of Keyman on the device (don't forget to reset user settings for the app; in Android Pie: Settings>Storage>Other apps>Keyman>Clear Data)
  2. Install test build APK off of TBD
  3. Accept all the Android permission requests for storage (applies to Android 6.0 Marshmallow onward)

User Testing

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

click to expand ## Various versions of Androids UI for these tests assume default English locale * GROUP_ANDROID_5 * GROUP_ANDROID_9 * GROUP_ANDROID_12 - **TEST_GET_STARTED:** This tests the "Get Started" menu from a fresh installation
Expand to see how to do and verify this 1. When the app loads, verify the "Get Started" menu is visible 2. From the "Get Started" menu, touch "Add a keyboard for your language" 3. Verify "Install Keyboard or Dictionary" menu appears 4. From the "Install Keyboard or Dictionary" menu, click the back arrow to return to "Get Started" 5. From the "Get Started" menu, touch "Enable Keyman as system-wide keyboard" 6. Verify Android system menus appear for controlling virtual keyboard 7. Scroll down and enable Keyman as a system keyboard. 8. On the "Attention" dialog, read and hit "OK" to dismiss the dialogs. 9. Touch the back button to return to "Get Started" 10. Verify "Enable Keyman as system-wide keyboard" now has a ticked checkbox. 11. From the "Get Started" menu, touch "Set Keyman as default keyboard" 12. On the "Change keyboard" dialog, select Keyman 13. Verify on the "Get Started" menu that "Set Keyman as default keyboard" now has a ticked checkbox. 14. From the "Get Started" menu, touch "More info" 15. Verify the Info page appears with a version string at the bottom of the page 16. Touch the back button (arrow) to return to "Get Started" 17. From the "Get Started" menu, untick the last option 'Show "Get Started" on startup' 18. Close the "Get Started" menu and exit the app. 19. From the Android device, start Keyman for Android 20. Verify the "Get Started" menu does not appear.
- **TEST_INSTALL_KD:** This menu is accessed via **Get Started** menu or the Settings button-->"Install Keyboard or Dictionary" 1. On the device, enable internet connectivity 2. Touch "Install from keyman.com" 3. Verify a new keyboard can be downloaded and selected 4. On the device, disable internet connectivity 5. From "Install Keyboard or Dictionary", touch "Install from keyman.com" 6. Verify device displays a message "Cannot connect to Keyman server!" 7. On the device, re-enable internet connectivity
- **TEST_KEYMAN_SETTTINGS:**
Expand to see how to do and verify this 1. From "Settings" --> "Installed Languages" 2. Select "English" --> English Settings 3. Select keyboard settings menu 4. Verify Info appears with keyboard version, help link, and QR code 5. If keyboard update is available, click on version and verify update 6. If help link is available, click on link 7. Verify keyboard help documentation appears 8. Return to English Settings menu 9. Select Dictionary --> English dictionary (MTNT) 10. Verify the Dictionary info appears 11. Return to English Settings menu 12. Turn on/off 'Enable Predictions' and validate each 13. Turn on/off 'Enable Corrections' and validate each. For example, when Corrections are enabled and context "tooo" should still suggest "too" 14. Validate that when both predictions and corrections are off, banner is not visible 15. Add/Remove dictionaries - validate 16. If multiple dictionaries are available, test swapping between them

- **TEST_SHARE:** Type and verify text can be shared to external app - **TEST_KEYMAN_BROWSER:** Keyman Browser ### 1. On default page, click on text field and set system keyboard to Keyman 2. Verify can type with Keyman as system keyboard 3. Close and reopen Keyman app
- **TEST_TEXT_SIZE:** Verify text can be rescaled from Text Size 16 to 72 - **TEST_CLEAR_TEXT:** Verify text can be cleared - **TEST_INFO:** 1. From the device, disable internet access 2. From the Keyman app menu, select "Info" 3. Verify Keyman for Android version appears at the bottom of the screen 4. Verify Keyman for Android help documentation is displayed. 5. From the device, enable internet access
- **TEST_INSTALL_UPDATES:** This menu option only appears when a language resource (keyboard or lexical-model) update is available
Expand to see how to do and verify this 1. Download an [old version of sil_cameroon_qwerty](https://downloads.keyman.com/keyboards/sil_cameroon_qwerty/6.0.4/sil_cameroon_qwerty.kmp) 2. On the device, set airplane mode on (turns off internet access) 3. On the device, go to settings, and manually set the date to 1 year in the past (e.g. May 2021) 4. Start Keyman for Android 5. From the Keyman Settings, install the local kmp file sil_cameroon_qwerty.kmp (will be an old version 6.0.4) 6. Close Keyman 7. On the device, go to settings and set the date to automatically update from internet 8. On the device, turn off airplane mode (turns on internet access) 9. On the device, go to settings and verify the date is updated 10. Launch Keyman for Android 11. Dismiss "Get Started" and wait for the catalog to update 12. After a while, there should be a notification of a keyboard update being available 13. At the bottom of the Keyman overflow menu, verify keyboard update is available 14. Click on "Install Updates" 15. Accept "Download" on the dialogue 16. Verify resource is updated after background download complete

- **TEST_CHANGE_DISPLAY_LANG:** This tests changing the display language for the App and assumes starting with English locale 1. From "Settings" --> "Display Language" 2. Select "Khmer" 3. From "Settings" --> Verify menu selections in Khmer 4. From the overflow menu --> clear text 5. Verify Keyman text field has Khmer prompt (for "Start typing here") 6. From "Settings" --> "Change Display Language" 7. If the string is translated, the first row should be "Default Locale" in Khmer 8. Select "English" 9. Verify the app reloads in English
- **TEST_ADJUST_K_HEIGHT:** This menu option allows the user to adjust the OSK height for the current orientation (portrait or landscape). The customized height is saved as a preference
Expand to see how to do and verify this 1. Have the device in portrait orientation 2. From "Settings" --> "Adjust keyboard height" 3. On the menu, drag the OSK height to change the keyboard height 4. Click back 5. Verify the OSK is refreshed to the selected height 6. From "Settings" --> "Adjust keyboard height" 7. Click the "reset to defaults" button and then click back 8. Verify the OSK reverts to the original height 9. Rotate the device to landscape orientation 10. From "Settings" --> "Adjust keyboard height" 11. On the menu, drag the OSK height to change the keyboard height 12. Click back 13. Verify the OSK is refreshed to the selected height 14. From "Settings" --> "Adjust keyboard height" 15. Click the "reset to defaults" button and then click back 16. Verify the OSK reverts to the original height 17. Rotate the device back to portrait orientation

- **TEST_SPACEBAR_CAPTION:** This menu option allows the user to adjust the label displayed on the keyboard spacebar
Expand to see how to do and verify this 1. Select the default SIL EuroLatin keyboard 2. From "Settings" --> Spacebar caption 3. On the menu, select "Language" 4. Click back until the OSK is displayed 5. Verify the spacebar label is "English" 6. From "Settings" --> Spacebar caption 7. On the menu, select "Keyboard" 8. Click back until the OSK is displayed 9. Verify the spacebar label is "EuroLatin (SIL)" 10. From "Settings" --> Spacebar caption 11. On the menu, select "Blank" 12. Click back until the OSK is displayed 13. Verify the spacebar label is blank 14. From "Settings" --> Spacebar caption 15. On the menu, select "Language+Keyboard" 16. Click back until the OSK is displayed 17. Verify the spacebar label is "English - EuroLatin (SIL)"

SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

click to expand ## Various versions of Androids UI for these tests assume default English locale * GROUP_ANDROID_5 * GROUP_ANDROID_9 * GROUP_ANDROID_12 ### In-app Keyboards These tests are run in the context of typing in the Keyman app
click to expand - **TEST_INAPP_LATIN_KEYBOARD_PORTRAIT:** English(EuroLatin SIL) In portrait orientation, verify OSK is visible and fills the width the bottom of the screen 1. Verify long-press `q` key works 2. Verify long-press `p` key works 3. Verify uppercase layer can be selected via `SHIFT` 4. Verify number layer can be selected via `123` 5. Verify long-press `1` key works 6. Verify long-press `0` key works 7. Verify backspace, space, and enter keys work 8. Verify *Khmer* -> *Khmer Angkor* keyboard can be added via Settings menu
- **TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE:** English(EuroLatin SIL) In portrait orientation, verify OSK is visible and fills the width the bottom of the screen 1. Verify long-press `q` key works 2. Verify long-press `p` key works 3. Verify uppercase layer can be selected via `SHIFT` 4. Verify number layer can be selected via `123` 5. Verify long-press `1` key works 6. Verify long-press `0` key works 7. Verify backspace, space, and enter keys work 8. Verify *Khmer* -> *Khmer Angkor* keyboard can be added via Settings menu
- **TEST_INAPP_LONG_PRESS:** English(EuroLatin SIL) Type on the OSK using the following scenarios and verify expected output: 1. Clicking a suggestion on the suggestion banner - should insert the suggestion 2. short-press a key and release - should insert the base key 3. long-press a key, select a long-press key, and release - should insert the long-press key 4. long-press a key, while keeping the finger down, move off the long-press options, and release - should **not** output 5. long-press a key, while keeping the finger down, move off the long-press options, then move back on a long-press option so it's highlighted, and release - should output the long-press key 6. quickly type a long paragraph (e.g. repeat the word "reply") - verify long-press keys don't get stuck (displayed when not touching a key)
- **TEST_INAPP_NON_LATIN_KEYBOARD:** Non-Latin script Keyboard 1. In the app, add/select a random non-Latin script keyboard 2. Verify OSK is visible 3. Test several keys and modifiers and verify Latin characters are not appearing.
- **TEST_INAPP_ROTATE_P-TO-L**: 1. With Keyman not active, load it in a portrait orientation. 2. Once loaded, rotate the device to a landscape orientation. - On Android devices, you may need to press something like this for the rotation to occur: - The test "passes" if the OSK rotates properly.
- **TEST_INAPP_ROTATE_L-TO-P**: 1. With Keyman not active, load it in a landscape orientation. 2. Once loaded, rotate the device to a portrait orientation. - On Android devices, you may need to press something like this for the rotation to occur: - The test "passes" if the OSK rotates properly.

### System Keyboards These tests are run with Keyman selected as a system keyboard, and in the context of typing in an external app (like Google Keep)
click to expand - **TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT:** English (EuroLatin SIL) In portrait orientation, verify OSK is visible and fills the width the bottom of the screen 1. Verify long-press `q` key works 2. Verify long-press `p` key works 3. Verify uppercase layer can be selected via `SHIFT` 4. Verify number layer can be selected via `123` 5. Verify long-press `1` key works 6. Verify long-press `0` key works 7. Verify backspace, space, and enter keys work 8. Verify can change to random non-Latin script keyboard via globe button
- **TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE:** English(EuroLatin SIL) In portrait orientation, verify OSK is visible and fills the width the bottom of the screen 1. Verify long-press `q` key works 2. Verify long-press `p` key works 3. Verify uppercase layer can be selected via `SHIFT` 4. Verify number layer can be selected via `123` 5. Verify long-press `1` key works 6. Verify long-press `0` key works 7. Verify backspace, space, and enter keys work 8. Verify *Khmer* -> *Khmer Angkor* keyboard can be added via Settings menu
- **TEST_SYSTEM_LONG_PRESS:** English(EuroLatin SIL) Type on the OSK using the following scenarios and verify expected output: 1. Clicking a suggestion on the suggestion banner - should insert the suggestion 2. short-press a key and release - should insert the base key 3. long-press a key, select a long-press key, and release - should insert the long-press key 4. long-press a key, while keeping the finger down, move off the long-press options, and release - should **not** output 5. long-press a key, while keeping the finger down, move off the long-press options, then move back on a long-press option so it's highlighted, and release - should output the long-press key 6. quickly type a long paragraph (e.g. repeat the word "reply") - verify long-press keys don't get stuck (displayed when not touching a key)
- **TEST_KEYBOARD_PICKER:** Keyboard Picker menu to switch input method This tests the Keyboard Picker menu which lists other system input methods at the bottom of the menu. These appear only in Keyman system keyboard 1. With only 1 Keyman keyboard installed and selected as a system keyboard, long-press and release the the globe button 2. Verify the bottom of the Keyboards picker menu lists other available input methods 3. Click on one of the other input methods 4. Verify the Keyman system keyboard switches to the selected input method 5. On the device's Settings, select Keyman as the input method 6. On the Keyman keyboard, short-press and release the globe button 7. Verify the previous input method is selected
- **TEST_SYSTEM_ROTATE_P-TO-L**: 1. With Keyman not active, load it in a portrait orientation. 2. Once loaded, rotate the device to a landscape orientation. - On Android devices, you may need to press something like this for the rotation to occur: - The test "passes" if the OSK rotates properly.
- **TEST_SYSTEM_ROTATE_L-TO-P**: 1. With Keyman not active, load it in a landscape orientation. 2. Once loaded, rotate the device to a portrait orientation. - On Android devices, you may need to press something like this for the rotation to occur: - The test "passes" if the OSK rotates properly.
- **TEST_EXT_BLUETOOTH_KEYBOARD:** External (Bluetooth) keyboard This test is run with an external keyboard (USB or bluetooth) connected to the device
Expand to see how to do and verify this 1. Start with English(EuroLatin SIL) as the selected keyboard 2. On the device's Settings, show the input method (OSK) 3. Verify can type with numbers, letters, and punctuation 4. Verify `caps lock` has no effect when typing 5. Verify long-press on a character repeats that character 6. Verify `tab` toggles to next field 7. Verify `enter` adds a newline 8. Verify `backspace` removes a character 9. Verify ctrl+tab brings up the keyboard picker menu 10. Use tab or arrow keys to navigate the keyboard picker menu 11. Select a non-Latin keyboard 12. Resume typing and verify new script appears
- **TEST_EXT_AZERTY_KEYBOARD:** External European AZERTY keyboard This test is run with an external European AZERTY keyboard paired to the device. It should contain the 102nd key `<>` on the bottom row next to the left-shift. 1. Start Keyman and install the French Basic keyboard 2. From the physical keyboard, type on the top letter row (azerty) 3. Verify `azerty` appears 4. Type on the 102nd key 5. Verify `<` appears 6. Type shift-102nd key 7. Verify `>` appears

SUITE_ANDROID_KNOWN_BUGS: drawn from a list of bugs

click to expand ## Various versions of Androids UI for these tests assume default English locale * GROUP_ANDROID_5 * GROUP_ANDROID_9 * GROUP_ANDROID_12 - **TEST_GLOBE_KEY** Holding backspace should not trigger a longpress on another random key ([7155](https://github.com/keymanapp/keyman/issues/7155)) 1. In the Keyman app with the default sil_euro_latin keyboard and lexical model enabled, start typing a word 2. Accept a suggestion 3. Type another letter 4. Hold the backspace key 5. The previous letter is deleted 6. Observe the longpress menu of the previous letter appearing 7. `FAIL` this test if the longpress menu of the previous letter appears - **TEST_SUGGESTION_NO_CAPITAL** Initial suggestions when starting a new field are not always Initial cased ([7165](https://github.com/keymanapp/keyman/issues/7165)) 1. In the Keyman app with the default sil_euro_latin keyboard and lexical model enabled, start typing some text (any text really) 2. Clear the text using the option from the top right menu and observe the suggestion banner 3. If the banner doesn't suggest words with capitalization, `FAIL` this test. - **TEST_BROKEN_SUGGESTION** selecting the "undo" suggestion breaks subsequent input ([7167](https://github.com/keymanapp/keyman/issues/7167)) When we select the "undo" suggestion that becomes available with Backspace after accepting a suggestion, on the next keystroke Keyman inserts a broken suggestion. 1. In Keyman for Android, type Prob. 2. Select `Problem` from banner. 3. Press Backspace, then select `"Prob"` (undoing the last suggestion). 4. Then press Spacebar. 5. If `Prob ` is shown, PASS this test; otherwise if `Prothe ` or `Problem `! - **TEST_SHIFT_KEY_STUCK** At start of input, pressing globe key, going back, leads to stuck shift key ([7169](https://github.com/keymanapp/keyman/issues/7169)) 1. In Keyman for Android with the default keyboard on, press and hold the globe key to open the keyboard menu 2. Press the `<` button on the top left of the screen or Android' back button at the bottom of the screen 3. If the `Shift` key is stuck and unusable, FAIL this test. - **TEST_REDUCED_TOAST_NOISE** - Verifies the number of Toast notifications are reduced ([7170](https://github.com/keymanapp/keyman/issues/7170)) 1. Install the PR build of Keyman for Android 2. Launch the app and observe the "Get Started" menu is displayed 3. Verify there's no Toast notification about checking the cloud catalog for updates 4. Dismiss the "Get Started" menu and go to Keyman settings --> Installed Languages 5. Click on "+" and search for "afade" language keyboard to install (install a keyboard that has no associated dictionary) 6. Install sil_cameroon_azerty and install the keyboard for the "Afade" language" 7. After the keyboard package is installed, verify no Toast notifications appear about checking for associated dictionary 8. Go to Keyman Settings --> Installed Languages --> Afade 9. On the Afade language settings menu, click the bottom "Dictionary - Check for available dictionary" 10. Verify a Toast notification appears about checking for associated dictionary to download 11. Wait a few minutes and verify no additional toast notifications appear 12. Go to Keyman settings --> Installed Languages 13. Click on "+" and search for khmer_angkor (install a keyboard that has an associated dictionary) 14. Install khmer_angkor 15. After the keyboard package is installed, verify: * no Toast notifications appear about "checking for associated dictionary" * two Toast notifications do appear about a "dictionary being installed in the background" and eventually "Resources successfully updated" (this may take a while depending on network lag)


SUITE_IOS_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

click to expand - GROUP_IPHONE_PORTRAIT: portrait mode in iPhone - GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone - GROUP_IPAD_PORTRAIT: portrait mode in iPad - GROUP_IPAD_LANDSCAPE: landscape mode in iPad ### Test Cases - **TEST_GET_STARTED**: Get Started - Verify touching **Add a keyboard for your language** pulls up the **keyboard search** - This should also be accessible as **Settings** > **Installed Languages** > **`+`**. - Verify touching **Enable Keyman as system-wide keyboard** pulls up Keyman's menu within iOS settings. (11.0+ devices) - If older versions of iOS are supported, ensure that the help page for doing it manually is properly linked. - Verify touching **More info** brings up an accurate version and the main table-of-contents help page. - This should also be accessible as **Info** from the main menu dropdown (on phones) or as the circled-i button (on tablets). - Set **Don't show again** to checked, close and re-open Keyman, and verify **Get Started** menu does not appear - **TEST_ADD_NEW_KEYBOARD**: Add New Keyboard This menu is accessed via **Get Started** menu or **Settings** menu - On the device, enable internet connectivity - Verify a new keyboard can be downloaded and selected, say "US Basic" - Return to the text input screen and confirm that it operates correctly. - Without changing selected language, confirm that it can be cleanly deleted via the Settings menu. - Confirm that another keyboard is correctly selected afterward. - **TEST_ADD_NEW_KEYBOARD_FROM_FILE**: - On the device, go to keyman.com on Safari - Search for "US Basic" - scroll down to keyboard details and tap to download the keyboard package (i.e. `.kmp` file) - open Files app and navigate to Download folder to find "basic_kbdus.kmp" and tap on it to start the installation processes - Keyman app should be pulled up - follow the screen to finish the keyboard installation - verity that the keyboard is correctly installed and working as expected - **TEST_SWITCH_KEYBOARD** - press on the globe key - the keyboard menu should be shown - press on a keyboard to switch to, say "EuroLatin (SIL)" - confirm that "EuroLatin (SIL)" is now selected and ready to be used - swich back to "US Basic" and confirm it does as expected - **TEST_TEXT_SIZE**: Text Size - Verify text can be rescaled from Text Size 9 to 72 - **TEST_CLEAR_TEXT**: Clear Text - Verify text can be cleared - **TEST_INFO**: Info - Verify that the Keyman iOS version is displayed; record the version - Verify that the Info menu options matches that of the **More Info...** option from the **Get Started** menu option. - Verify that the Keyman iOS version matches the version string seen at (Home Screen) > [iOS] Settings > Keyman. - **TEST_SETTINGS**: Settings - Installed Languages ... Add Language - Select an installed language ... Add (another) Keyboard - Should automatically perform a filtered search based upon that language's BCP-47 code, i.e. for "English" it should be seen as shown in the screenshot below. - Select an installed language ... Dictionary - Turn on/off 'Enable Predictions' and validate each - Turn on/off 'Enable Corrections' and validate each - Validate that when both predictions and corrections are off, banner is not visible - Add/Remove dictionaries - validate - If multiple dictionaries are available, test swapping between them - **TEST_EUROLATIN**: `English (Eurolatin)` in-app - In portrait orientation, verify that the OSK appears and fills the width the bottom of the screen - Verify long-press `q` key works - Verify long-press `k` key works - Verify that any predictive suggestions generated feel 'reasonable' (based upon 'k', the selected subkey, or one of its neighbors) - Verify uppercase layer can be selected via `SHIFT` - Verify number layer can be selected via `123` - Verify long-press `1` key works - Verify long-press `0` key works - Verify backspace, space, and enter keys work Note that EuroLatin2, a highly customized keyboard explicitly designed for mobile, is not found; hence EuroLatin (SIL) is used for testing this instead. - **TEST_US_BASIC**: English: `US Basic` in-app - Verify that the **US Basic** keyboard can be downloaded through Settings > Installed Languages > English > `+` - This should automatically bring up search results for **English** keyboards. - When the package-installer launches, English should be the one pre-selected language. - Install the package for both English and Tagalog by selecting both of them in the installation processes. - Set **English: US Basic** as the active in-app keyboard. - Verify that predictive text suggestions appear. - Set **Tagalog: US Basic** as the active in-app keyboard. - Verify that predictive text suggestions do _not_ appear. - Verify that long-press `123` key works, presenting options for a currency layer and a symbol layer. - Select the currency layer, then press the `Ā£` key. - Verify that the keyboard outputs correctly and returns to the default layer. - Go to the symbol layer and press the `Ā©` key. - Verify that the keyboard outputs correctly and remains on the symbol layer. - Revert to the default layer and ensure basic key inputs work. - Long-press `e` on the default layer and select a subkey. - Verify that the selected key produces the correct output. - From the `Settings > Languages > English > US Basic` menu, follow the "Help link" and ensure it displays appropriate help. - (Should work both online and offline) - From the `Settings > Languages > English > US Basic` menu, scan the QR code with a phone and test that it links to the current version of that keyboard's public download page on keyman.com. - Returning to the app, uninstall the **US Basic** keyboard via Settings for Tagalog. - Set **English: US Basic** as the active in-app keyboard. - Verify that the keyboard appears correctly and ensure basic key inputs work. - This is to ensure that uninstalling for one language doesn't adversely affect installations of the same resource for another language. - Uninstall the **US Basic** keyboard via Settings for English.

SUITE_IOS_SYSTEM_KEYBOARD:

click to expand - GROUP_IPHONE_PORTRAIT: portrait mode in iPhone - GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone - GROUP_IPAD_PORTRAIT: portrait mode in iPad - GROUP_IPAD_LANDSCAPE: landscape mode in iPad Test the following after setting Keyman as a system keyboard. Test Cases - **TEST_BASIC_USAGE**: ensure that the keyboards functions as expected on all layers. Please test on multiple keyboards, i.e. EuroLatin (SIL), Cameroon QWERTY keyboard and Khmer Angkor. - **TEST_LONG_PRESS**: ensure that longpress on all applicable layers are working as intended - **TEST_GLOBE_KEY**: ensure that a short press on the globe key switch to the next keyboard and a longpress on it open the keyboard menu where you can select a keyboard of your choice
keymanapp-test-bot[bot] commented 1 year ago

User Test Results

Test specification and instructions

šŸŸ„ SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

āœ… SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

42 tests in 3 groups PASSED * āœ… GROUP_ANDROID_5:
14 tests PASSED - āœ… **TEST_INAPP_LATIN_KEYBOARD_PORTRAIT ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527176926))**: Verified that the long-press key is working on both the layers (ie., base and shift and number). Also, verified that the backspace and enter key are working as expected. - āœ… **TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527176926))**: Verified that the long-press key is working on both the layers (ie., base and shift and number). Also, verified that the backspace and enter key are working as expected. - āœ… **TEST_INAPP_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527176926))**: Verified the Short Press key and Long Press key functionality in the OSK (EuroLatin SIL) are working as expected. - āœ… **TEST_INAPP_NON_LATIN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527176926))**: Verified that the non-latin script (eg., khmer) is visible and working in different layers. - āœ… **TEST_INAPP_ROTATE_P-TO-L ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527176926))**: Verified that the OSK rotates properly (from Portrait to Landscape orientation) - āœ… **TEST_INAPP_ROTATE_L-TO-P ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527176926))**: Verified that the OSK rotates properly (from Landscape to Portrait orientation) - āœ… **TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Verified that the long-press key is working on both the layers (ie., base and shift and number). Also, verified that the backspace and enter key are working as expected. - āœ… **TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Verified that the long-press key is working on both the layers (ie., base and shift and number). Also, verified that the backspace and enter key are working as expected. - āœ… **TEST_SYSTEM_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Verified the Short Press key and Long Press key functionality in the OSK (EuroLatin SIL) are working as expected. - āœ… **TEST_KEYBOARD_PICKER ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Verified that the Keyboard Picker menu which lists other system input methods at the bottom of the menu. - āœ… **TEST_SYSTEM_ROTATE_P-TO-L ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Verified that the OSK rotates properly (from Portrait to Landscape orientation) - āœ… **TEST_SYSTEM_ROTATE_L-TO-P ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Verified that the OSK rotates properly (from Landscape to Portrait orientation) - āœ… **TEST_EXT_BLUETOOTH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Tested with external keyboards like USB and Bluetooth connected devices and verified all the tests that have been mentioned in the test case (Test_Ext_Bluetooth_Keyboard). - āœ… **TEST_EXT_AZERTY_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527306764))**: Tested with external European AZERTY keyboard paired to my mobile device. Verified that typing 'azerty' letters from an external keyboard produces the same letters in the keyman text input screen. Pressing 102nd key shows < symbol and pressing shift+102 key shows > symbol.
* āœ… GROUP_ANDROID_9:
14 tests PASSED - āœ… **TEST_INAPP_LATIN_KEYBOARD_PORTRAIT ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the OSK is visible and fills the width the bottom of the screen, inthe portrait orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the OSK is visible and fills the width the bottom of the screen, inthe landscape orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_INAPP_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that short press key and long press key functionality is working as expected. Also, verified that typing 'reply' word in the keyman app quickly, did not stuck the keyboard operations. - āœ… **TEST_INAPP_NON_LATIN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the Non-Latin script keyboard (eg., Khmer Angkor) is working as expected. - āœ… **TEST_INAPP_ROTATE_P-TO-L ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the Keyboard orientation did not get affected after rotating it from Portrait to Landscape. - āœ… **TEST_INAPP_ROTATE_L-TO-P ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the Keyboard orientation did not get affected after rotating it from Landscape to Portrait. - āœ… **TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the OSK is visible and fills the width the bottom of the screen, in the portrait orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the OSK is visible and fills the width the bottom of the screen, in the landscape orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_SYSTEM_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that short press key and long press key functionality is working as expected. Also, verified that typing 'reply' word in the keyman app quickly, did not stuck the keyboard operations. - āœ… **TEST_KEYBOARD_PICKER ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the Non-Latin script keyboard (eg., Khmer Angkor) is working as expected. - āœ… **TEST_SYSTEM_ROTATE_P-TO-L ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the Keyboard orientation did not get affected after rotating it from Portrait to Landscape. - āœ… **TEST_SYSTEM_ROTATE_L-TO-P ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the Keyboard orientation did not get affected after rotating it from Landscape to Portrait. - āœ… **TEST_EXT_BLUETOOTH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1534672010))**: Retested with the attached PR build (17.0.98-alpha-test-8544) in my Android Mobile device along with external (USB) keyboard and it is working fine except the 'tab' key. Is this key is optional? - āœ… **TEST_EXT_AZERTY_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))**: Verified that the External Azerty (bluetooth) keyboard is working as expected. ([notes](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508354788))
* āœ… GROUP_ANDROID_12:
14 tests PASSED - āœ… **TEST_INAPP_LATIN_KEYBOARD_PORTRAIT ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the OSK is visible and fills the width the bottom of the screen, in the portrait orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the OSK is visible and fills the width the bottom of the screen, in the landscape orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_INAPP_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that short press key and long press key functionality is working as expected. Also, verified that typing 'reply' word in the keyman app quickly, did not stuck the keyboard operations. - āœ… **TEST_INAPP_NON_LATIN_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the Non-Latin script keyboard (eg., Khmer Angkor) is working as expected. - āœ… **TEST_INAPP_ROTATE_P-TO-L ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the Keyboard orientation did not get affected after rotating it from Portrait to Landscape. - āœ… **TEST_INAPP_ROTATE_L-TO-P ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the Keyboard orientation did not get affected after rotating it from Landscape to Portrait. - āœ… **TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the OSK is visible and fills the width the bottom of the screen, in the portrait orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the OSK is visible and fills the width the bottom of the screen, in the landscape orientation. Also, verified long pressing alphanumeric keys shows the expected output. Able to install Khmer Angkor keyboard via Settings menu. - āœ… **TEST_SYSTEM_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that short press key and long press key functionality is working as expected. Also, verified that typing 'reply' word in the keyman app quickly, did not stuck the keyboard operations. - āœ… **TEST_KEYBOARD_PICKER ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the Non-Latin script keyboard (eg., Khmer Angkor) is working as expected. - āœ… **TEST_SYSTEM_ROTATE_P-TO-L ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the Keyboard orientation did not get affected after rotating it from Portrait to Landscape. - āœ… **TEST_SYSTEM_ROTATE_L-TO-P ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the Keyboard orientation did not get affected after rotating it from Landscape to Portrait. - āœ… **TEST_EXT_BLUETOOTH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1534672010))**: Retested with the attached PR build (17.0.98-alpha-test-8544) in my Android Mobile device along with external (USB) keyboard and it is working fine except the 'tab' key. Is this key is optional? - āœ… **TEST_EXT_AZERTY_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508467833))**: Verified that the External Azerty (bluetooth) keyboard is working as expected.

āœ… SUITE_ANDROID_KNOWN_BUGS: drawn from a list of bugs

15 tests in 3 groups PASSED * āœ… GROUP_ANDROID_5:
5 tests PASSED - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527402985))**: Verified that holding backspace key did not trigger the longpresskey or any other key. - āœ… **TEST_SUGGESTION_NO_CAPITAL ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527402985))**: After clearing the text, the banner shows the suggest words with capitalization. - āœ… **TEST_BROKEN_SUGGESTION ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1534592558))**: Retested and verified that the word "Prob" only appears on the text input screen. Seems to be working fine. - āœ… **TEST_SHIFT_KEY_STUCK ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527402985))**: Verified that the Shift key dig not get stuck at any point. - āœ… **TEST_REDUCED_TOAST_NOISE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1527402985))**: Verified that the Toast notifications are reduced and appeared for some of the functionalities. Seems to be working as expected.
* āœ… GROUP_ANDROID_9:
5 tests PASSED - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508607363))**: Verified that holding backspace key should not trigger a longpress on another random key. - āœ… **TEST_SUGGESTION_NO_CAPITAL ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1539610846))**: Retested with the updated Keyman build (17.0.98-alpha-test-8544) in Android 9.0 / API 28 emulator and here is my observation: 1. After clearing the text using Clear text option, the banner suggest words with capitalization. Seems to be working as expected. - āœ… **TEST_BROKEN_SUGGESTION ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1534635712))**: Retested and verified that the word "Prob" only appears on the text input screen. Seems to be working fine. - āœ… **TEST_SHIFT_KEY_STUCK ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508607363))**: Pressing globe key, going back to home screen did not stuck the shift key. Seems to be working fine. - āœ… **TEST_REDUCED_TOAST_NOISE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508607363))**: Verified that the number of Toast notifications are reduced as per the instructions.
* āœ… GROUP_ANDROID_12:
5 tests PASSED - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508660103))**: Verified that holding backspace key should not trigger a longpress on another random key. - āœ… **TEST_SUGGESTION_NO_CAPITAL ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1539610846))**: Retested with the updated Keyman build (17.0.98-alpha-test-8544) in Android 12.0 / API 31 emulator and here is my observation: 1. After clearing the text using Clear text option, the banner suggest words with capitalization. Seems to be working as expected. - āœ… **TEST_BROKEN_SUGGESTION ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1534635712))**: Retested and verified that the word "Prob" only appears on the text input screen. Seems to be working fine. - āœ… **TEST_SHIFT_KEY_STUCK ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508660103))**: Pressing globe key, going back to home screen did not stuck the shift key. Seems to be working fine. - āœ… **TEST_REDUCED_TOAST_NOISE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1508660103))**: Verified that the number of Toast notifications are reduced as per the instructions.

šŸŸ§ SUITE_IOS_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

āœ… SUITE_IOS_SYSTEM_KEYBOARD:

12 tests in 4 groups PASSED * āœ… GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
3 tests PASSED - āœ… **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1533005075))**: Retested with the updated build (17.0.98-alpha-test-8544) in iOS 16.2 / iPhone 14 Pro Max simulator and here is my observation: Installed 3 different keyboards and tested it in Safari browser and it is working as expected. - āœ… **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1533005075))**: Long pressing (sil_euro_latin or khmer_angkor) keyboards outputs the subkey menus on the browser. Seems to be working as expected. - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1531516572))**: Short press (in the Keyman In-App) and Long press keys (Keyman as the System Keyborad) are working fine.
* āœ… GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
3 tests PASSED - āœ… **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1533005075))**: Retested with the updated build (17.0.98-alpha-test-8544) in iOS 16.2 / iPhone 14 Pro Max simulator and here is my observation: Installed 3 different keyboards and tested it in Safari browser and it is working as expected. - āœ… **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1533005075))**: Long pressing (sil_euro_latin or khmer_angkor) keyboards outputs the subkey menus on the browser. Seems to be working as expected. - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1531535167))**: Short press (in the Keyman In-App) and Long press keys (Keyman as the System Keyborad) are working fine.
* āœ… GROUP_IPAD_PORTRAIT: portrait mode in iPad
3 tests PASSED - āœ… **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1533065739))**: Retested with the updated build (17.0.98-alpha-test-8544) in iOS 16.2 / iPad (10th generation) simulator and here is my observation: Installed 3 different keyboards and tested it in Safari browser and it is working as expected. - āœ… **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1535829094))** - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1531556460))**: Short press (in the Keyman In-App) and Long press keys (Keyman as the System Keyborad) are working fine.
* āœ… GROUP_IPAD_LANDSCAPE: landscape mode in iPad
3 tests PASSED - āœ… **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1533065739))**: Retested with the updated build (17.0.98-alpha-test-8544) in iOS 16.2 / iPad (10th generation) simulator and here is my observation: Installed 3 different keyboards and tested it in Safari browser and it is working as expected. - āœ… **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1535829094))** - āœ… **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/pull/8544#issuecomment-1531556460))**: Short press (in the Keyman In-App) and Long press keys (Keyman as the System Keyborad) are working fine.
Retesting Template ``` @keymanapp-test-bot retest SUITE_ANDROID_MENU_FUNCTIONALITY GROUP_ANDROID_5 TEST_KEYMAN_SETTTINGS SUITE_IOS_MENU_FUNCTIONALITY GROUP_IPAD_PORTRAIT TEST_US_BASIC GROUP_IPAD_LANDSCAPE TEST_US_BASIC ```

Test Artifacts

jahorton commented 1 year ago

Re: the iOS build failure:

[08:21:09 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2535&logView=flowAware&linesState=2535)
  Copying Keyman Engine samples into upload/17.0.67/keyman-engine-ios-17.0.67.zip...

[08:21:09 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2536&logView=flowAware&linesState=2536)
  Copying Keyman IPA build/Build/Products/Release-iphoneos/Keyman.ipa => upload/17.0.67/keyman-ios-17.0.67.ipa...

[08:21:09 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2537&logView=flowAware&linesState=2537)
  Zipping Keyman simulator artifact build/Build/Products/Release-iphonesimulator/Keyman.app => upload/17.0.67/keyman-ios-simulator-17.0.67.app.zip...

[08:21:09 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2538&logView=flowAware&linesState=2538)
  /Users/mcdurdin/buildAgent/work/99b311828f4ee7c/keyman/ios/upload/17.0.67/keyman-ios-simulator-17.0.67.app.zip

[08:21:09 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2539&logView=flowAware&linesState=2539)
  Copying FirstVoices IPA ../oem/firstvoices/ios/build/Build/Products/Release-iphoneos/FirstVoices.ipa => upload/17.0.67/firstvoices-ios-17.0.67.ipa...

[08:21:09 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2540&logView=flowAware&linesState=2540)
  Zipping FirstVoices simulator artifact ../oem/firstvoices/ios/build/Build/Products/Release-iphonesimulator//FirstVoices.app => upload/17.0.67/keyman-ios-simulator-17.0.67.app.zip...

[08:21:10 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2541&logView=flowAware&linesState=2541)
  stat: invalid option -- '%'

[08:21:10 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2542&logView=flowAware&linesState=2542)
  Try 'stat --help' for more information.

[08:21:10 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2543&logView=flowAware&linesState=2543)
  Process exited with code 1

[08:21:11 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2545&logView=flowAware&linesState=2545)
  Process exited with code 1 (Step: Use prep-release script to capture build artifacts (Command Line))

[08:21:11 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2546&logView=flowAware&linesState=2546)
  Step Use prep-release script to capture build artifacts (Command Line) failed

[08:21:11 ](https://build.palaso.org/buildConfiguration/Keyman_iOS_TestPullRequests/375628?buildTab=log&focusLine=2548&logView=flowAware&linesState=2548)
Step 6/6: Upload to TestFlight (PR area) (Command Line)

Looks like recent updates to the build agents caused the artifact upload to fall over? At least the build completed fine up 'til then.

mcdurdin commented 1 year ago

Looks like recent updates to the build agents caused the artifact upload to fall over? At least the build completed fine up 'til then.

Yes, upgrade of macOS meant that coreutils stat was being used instead of system one. Yay. Fixed in #8498.

(Ctrl/Cmd + Shift + V pastes plain text from build logs so you don't get the links embedded on each line -- much more readable)

jahorton commented 1 year ago

Now that I've folded in the fix from #8498 (after the accidental merge fiasco), the iOS app should at least build now. I've gone ahead and copied over relevant tests from the iOS acceptance-testing suite in anticipation of that.

mcdurdin commented 1 year ago
  • Sourcemapping woes: mostly because of the KMW build product getting renamed when it's copied into the Android engine library's asset bundle. (Sourcemaps are often connected by a relative link in comment at the end that relies on the original name of the build output - pre-copy.)

I see no reason for us to rename the KMW build product for Android Engine -- this should be a simple fix, shouldn't it?

mcdurdin commented 1 year ago
  • I was a bit surprised at the size of the build product - it clocks in at 461 kB

This will need to be addressed -- it's very not good to have a 70KB increase in file size for zero user-visible functionality improvement!

  • I did some investigation with the minifyIdentifiers flag, which was disabled for the result above.

Definitely don't enable this flag in this epic. It adds another layer of fragility (in that is really hard to reliably and thoroughly test), and this epic already changes so many things which are going to take quite a while to shake out completely.

jahorton commented 1 year ago
  • I was a bit surprised at the size of the build product - it clocks in at 461 kB

This will need to be addressed -- it's very not good to have a 70KB increase in file size for zero user-visible functionality improvement!

After a fair bit of analysis of the build products and tinkering with minification settings with the most likely-to-be-relevant modules... it's because of the change in minification. To be precise, that minifyIdentifiers flag we can feed esbuild? Disabling that cancels part of the minification we've been having from Google Closure.

As an example, let's take the OSKView class, which has numerous fields and methods. With Closure, and with the option enabled, we get code like...

var OSKView = function() {} // class construction method
var o = OSKView;
o.prototype.show = function(param) {} // defines OSKView.prototype.show

// ...

let oskInstance = new o(); // within the appropriate method, of course.

Without it...

var OSKView = function() {} // class construction method
OSKView.prototype.show = function(p1) {} // defines OSKView.prototype.show

// ...

let oskInstance = new OSKView(); // within the appropriate method, of course.

Fields and property names aren't being altered; it's just locally-scoped names... and there are a lot of them due to how modules operate. Parameter names for functions, as well as local-scope variables, are similarly renamed, as they only exist within their functions' scopes.

A point of comparison: my initial suspicions were on engine/osk, as I was surprised at the size of its isolated bundle. But, to confirm my analysis above... note that only enabling that one minification flag (minifyIdentifiers) for the engine/osk component shrinks its bundled-module form from 328 KB to 271 KB. No whitespace elimination or syntax minification. That's 57 KB, the majority of the excess filesize noted in the quoted comment above... and is still for a submodule, not the complete compiled package. (To be clear, there are also parts of KMW's build product that simply don't exist for app/webview, so we'd expect more savings to be found elsewhere. This build product should be notably smaller, ideally speaking.)

So... that's why KMW's size grew here. We lost parts of minification that were previously enabled under Google Closure. Whitespace minification is obviously 100% safe and is a larger win, which is why we're at least still on the same order of magnitude, but it does make a notable difference in final filesizes.

There's also a fair bit of cruft from certain TypeScript transformations used to target ES5: a major offender here is the __extends function used for class inheritance. It's only defined a single time in namespaced-TS compilations, largely because all the linking is done by TS. That's not the case for esbuild-based bundling though... and app/webview has that transform-function defined thirty separate times - likely once for each module file in need of the function. It's not a large method, mind you, but still.

image

The function, in minified form:

var __extends=function(){var extendStatics=function(d,b){return extendStatics=Object.setPrototypeOf||{__proto__:[]}instanceof Array&&function(d2,b2){d2.__proto__=b2}||function(d2,b2){for(var p in b2)Object.prototype.hasOwnProperty.call(b2,p)&&(d2[p]=b2[p])},extendStatics(d,b)};return function(d,b){if(typeof b!="function"&&b!==null)throw new TypeError("Class extends value "+String(b)+" is not a constructor or null");extendStatics(d,b);function __(){this.constructor=d}d.prototype=b===null?Object.create(b):(__.prototype=b.prototype,new __)}}()

The other copies of the transform get named __extends2, __extends3, and so forth. Identical implementations, the lot of 'em.

Saving just that var __extends codeblock to an isolated file clocks in at 546 bytes. Multiply by 30... and that's about 16 KB. 16.38 KB if using 1000 instead of 1024 as the "kilo" factor... and I don't know of a way to get esbuild to recognize them all as the same definition. Sadly, esbuild doesn't transpile JS code for more-recent ES-specs down to ES5 - and quite explicitly so - which would otherwise be the obvious way to avoid the problem.

mcdurdin commented 1 year ago

a major offender here is the __extends function used for class inheritance

Does this help? https://stackoverflow.com/a/38390202/1836776

jahorton commented 1 year ago

Well, I just tried out the iOS build artifact on my personal phone as a smoke test. Yeah... of course I missed something. I got presented with a beautiful blank keyboard. As a related point, there appear to be no corresponding error logs in Sentry.

... and yeah, looks like the coreutils change may be affecting the iOS build - the resource-copying step in app/webview/build.sh is acting differently on my mac than on my windows PC, matching what's stated with #8565. Yaaaaaay.

jahorton commented 1 year ago

Alright... the iOS build is now patched up and the keyboard is now loading and interactive.

That said, I've come to realize that keyboard fonts aren't being loaded properly at the moment - both for iOS and for Android. I only just realized it because the default font was far more different on iOS.

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_9:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_9:

bharanidharanj commented 1 year ago

SUITE_IOS_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPHONE_PORTRAIT: portrait mode in iPhone

..in iPhone 14 Pro Max Simulator

bharanidharanj commented 1 year ago

SUITE_IOS_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPAD_PORTRAIT: portrait mode in iPad

..in iPad (10th generation) Simulator

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_9:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_9:

jahorton commented 1 year ago

Huh, I wonder what late change broke the CSS inclusion for the iOS app. I'll have to get on that, but at least it checks out on my personal device.

At least there's still plenty to test for the Android app.

jahorton commented 1 year ago

The iOS issue reported above is nigh-certainly related to the same issue addressed by #8565. I mentioned in this comment (https://github.com/keymanapp/keyman/pull/8544#issuecomment-1502649520) that'd I'd fixed it... but apparently the setup between my local dev machine and the build agents is different, so the OSK's resources aren't landing in the correct location. (So, my local cp isn't matching the one on the build agent, or something similar.)

That's what the OSK looks like when the stylesheet used for keyboard layout formatting goes missing.

sigh

I guess we're not done with addressing those build-agent update side-effects yet.

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_9:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_12:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_9:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_12:

jahorton commented 1 year ago

Interestingly, https://github.com/keymanapp/keyman/pull/8544/commits/9b00bdac99601244e2878ba76b35ac7a6fb9b335 did not fix the iOS build - I'm still missing the OSK's CSS in the TestFlight upload.

Hmm. It's not missing in the builds based on master, though. That's probably worth a compare-and-contrast look.

jahorton commented 1 year ago

@keymanapp-test-bot retest SUITE_ANDROID_MENU_FUNCTIONALITY GROUP_ANDROID_5 , SUITE_ANDROID_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_5 , SUITE_ANDROID_KNOWN_BUGS GROUP_ANDROID_5

I've identified and fixed the causes behind the blank-keyboard issue that previously arose, as well as a few other Android 5 compatibility issues. These tests should no longer be blocked by a blank keyboard bug.

jahorton commented 1 year ago

@mcdurdin Looks like we have a user test-bot issue here; the test bot isn't properly processing the command provided by my previous comment. Only the last suite-group set is set to "retest". (Without the space, none are marked for retesting.)

Contrast with how https://github.com/keymanapp/keyman/wiki/User-Testing-Workflows#retesting suggests the comment should be parsed.

So, to ensure all are properly marked for retesting...

@keymanapp-test-bot retest SUITE_ANDROID_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_5

jahorton commented 1 year ago

And...

@keymanapp-test-bot retest SUITE_ANDROID_MENU_FUNCTIONALITY GROUP_ANDROID_5

jahorton commented 1 year ago

An update re: the iOS problem - I've yet to pin down the exact cause, but it appears to be strongly related to "debug" vs "release" configurations. When I build locally and deploy to my personal device from Xcode - so, "debug" config - everything works and I can inspect via Safari's develop menu from a Mac. But when I download the build artifact pushed to TestFlight - a "release" build - the stylesheet isn't being linked correctly... and I cannot inspect via Safari's develop menu. The stylesheet file exists in the proper location within the artifacts uploaded and pushed to TestFlight, so the file should be perfectly available for both configurations.

That said, a release config within Simulator doesn't suffer the same problem... but Simulator tends to get extra privileges in this regard.

These details are probably enough to help narrow the cause down. Just gotta pursue that thread now.

Also, for what it's worth, I've noticed that the standard OSK font that includes our special symbols is also not being loaded correctly when the stylesheet fails to load. (You can see random emoji in that stack.)

Also, all of these details only affect the ES module feature branch. The existing master doesn't have this stylesheet issue.

Maybe it has something to do with path configuration? I wonder if app-sandboxing effects present a notable difference within a hosted WebView in terms of window.location.href or similar.

In fact...

New:

https://github.com/keymanapp/keyman/blob/30da35c5a2596e185f48fdfebbb7b1fa790a4e26/web/src/app/webview/src/debug-main.ts#L4-L7

Which indirectly triggers...

https://github.com/keymanapp/keyman/blob/30da35c5a2596e185f48fdfebbb7b1fa790a4e26/web/src/engine/paths/src/pathConfiguration.ts#L38-L42

Old:

https://github.com/keymanapp/keyman/blob/f9b3b20ca5959334b261a281461a573d8a4dfd92/web/src/engine/main/kmwbase.ts#L708-L723

jahorton commented 1 year ago

Looks like https://github.com/keymanapp/keyman/pull/8544/commits/632a7762532d63fe1d3dd1b3da73cdace56c119f wasn't enough to correct the iOS release-build / TestFlight issue where the stylesheet & fonts aren't being properly applied.

(Referencing https://github.com/keymanapp/keyman/pull/8544#issuecomment-1510746538.)

jahorton commented 1 year ago

Yay, the keyboard is now operable on the latest TestFlight build. (Whew.)

Accordingly...

@keymanapp-test-bot retest SUITE_IOS_MENU_FUNCTIONALITY GROUP_IPHONE_PORTRAIT TEST_GET_STARTED

jahorton commented 1 year ago

Also...

@keymanapp-test-bot retest SUITE_IOS_MENU_FUNCTIONALITY GROUP_IPAD_PORTRAIT TEST_GET_STARTED

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

Test Results

SUITE_ANDROID_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_ANDROID_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_5:

bharanidharanj commented 1 year ago

SUITE_IOS_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPHONE_PORTRAIT: portrait mode in iPhone

jahorton commented 1 year ago

I've just pushed a few fixes up. It's end of day, so I'll go about doing the test bot instructions tomorrow, I suppose. Gotta confirm build success first, anyway.

bharanidharanj commented 1 year ago

Test Results

SUITE_IOS_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone