keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web
https://keyman.com/
Other
403 stars 112 forks source link

test(iOS): Regression Test for Alpha 16.0.48 #7093

Closed MakaraSok closed 2 years ago

MakaraSok commented 2 years ago

Keyman for iPhone and iPad Acceptance Test Procedures

  1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
  2. Copy these tests into a new issue (for tier transitions) or a comment on the PR.
  3. When annotating results, use > at the start of a new line under the checkbox to note the result. (For an example, see #3973).

User Testing

Gather Assets for Testing

At this time, Keyman for iOS does not support external keyboard input.

Setup Steps

  1. Uninstall previous version of Keyman on the device (on a simulator, "Erase All Content and Settings..." under Device menu)
  2. Install test build IPA (on a simulator, "iOS Simulator Image")
  3. Disable internet connectivity of the device
  4. Start Keyman and select Add a keyboard for your language
  5. Verify error message "Could not reach Keyman server!" or "The Internet connection appears to be offline." is displayed
  6. Click OK to dismiss. Close Keyman app
  7. On device, enable internet connectivity
  8. Start Keyman
  9. On initial Get Started menu, select "Enable Keyman as system-wide keyboard" and do any necessary system steps to enable Keyman


SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

Test Cases

Note that EuroLatin2, a highly customized keyboard explicitly designed for mobile, is not found; hence EuroLatin (SIL) is used for testing this instead.


SUITE_SYSTEM_KEYBOARD:

Test the following after setting Keyman as a system keyboard.

Test Cases

SUITE_SWITCH_LOCALE

Test Cases


[TODO] SUITE_PREDICTIVE_TEXT: Predictive Text

keymanapp-test-bot[bot] commented 2 years ago

User Test Results

Test specification and instructions

⬜ SUITE_INSPECT: Visual Inspection of Alpha

✅ SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

48 tests in 4 groups PASSED * ✅ GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
12 tests PASSED - ✅ **TEST_GET_STARTED ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD_FROM_FILE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_SWITCH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_SHARE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_KEYMAN_BROWSER ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_TEXT_SIZE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_CLEAR_TEXT ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_INFO ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_SETTINGS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_EUROLATIN ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_US_BASIC ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1225819405))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected.
* ✅ GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
12 tests PASSED - ✅ **TEST_GET_STARTED ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD_FROM_FILE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_SWITCH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_SHARE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_KEYMAN_BROWSER ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_TEXT_SIZE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_CLEAR_TEXT ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_INFO ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_SETTINGS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_EUROLATIN ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_US_BASIC ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1226875346))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected.
* ✅ GROUP_IPAD_PORTRAIT: portrait mode in iPad
12 tests PASSED - ✅ **TEST_GET_STARTED ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD_FROM_FILE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_SWITCH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_SHARE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_KEYMAN_BROWSER ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_TEXT_SIZE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_CLEAR_TEXT ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_INFO ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_SETTINGS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_EUROLATIN ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_US_BASIC ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected.
* ✅ GROUP_IPAD_LANDSCAPE: landscape mode in iPad
12 tests PASSED - ✅ **TEST_GET_STARTED ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_ADD_NEW_KEYBOARD_FROM_FILE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_SWITCH_KEYBOARD ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_SHARE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_KEYMAN_BROWSER ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_TEXT_SIZE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_CLEAR_TEXT ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_INFO ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_SETTINGS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_EUROLATIN ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_US_BASIC ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227073438))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected.

✅ SUITE_SYSTEM_KEYBOARD:

12 tests in 4 groups PASSED * ✅ GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
3 tests PASSED - ✅ **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227169282))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227169282))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227169282))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected.
* ✅ GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
3 tests PASSED - ✅ **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227169282))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227169282))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected. - ✅ **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227169282))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and it is working as expected.
* ✅ GROUP_IPAD_PORTRAIT: portrait mode in iPad
3 tests PASSED - ✅ **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227251593))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227251593))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227251593))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected.
* ✅ GROUP_IPAD_LANDSCAPE: landscape mode in iPad
3 tests PASSED - ✅ **TEST_BASIC_USAGE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227251593))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_LONG_PRESS ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227251593))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected. - ✅ **TEST_GLOBE_KEY ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227251593))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and it is working as expected.

✅ SUITE_SWITCH_LOCALE:

8 tests in 4 groups PASSED * ✅ GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
2 tests PASSED - ✅ **TEST_KM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227272995))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and the UI language of the Keyman has been changed to Khmer Successfully. - ✅ **TEST_AM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227272995))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and the UI language of the Keyman has been changed to Amharic Successfully.
* ✅ GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
2 tests PASSED - ✅ **TEST_KM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227272995))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and the UI language of the Keyman has been changed to Khmer Successfully. - ✅ **TEST_AM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227272995))**: Tested this in iOS 15.5 / iPhone 13 Pro Max Simulator and the UI language of the Keyman has been changed to Amharic Successfully.
* ✅ GROUP_IPAD_PORTRAIT: portrait mode in iPad
2 tests PASSED - ✅ **TEST_KM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227359748))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and the UI language of the Keyman has been changed to Khmer Successfully. - ✅ **TEST_AM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227359748))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and the UI language of the Keyman has been changed to Amharic Successfully.
* ✅ GROUP_IPAD_LANDSCAPE: landscape mode in iPad
2 tests PASSED - ✅ **TEST_KM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227359748))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and the UI language of the Keyman has been changed to Khmer Successfully. - ✅ **TEST_AM_LOCALE ([PASSED](https://github.com/keymanapp/keyman/issues/7093#issuecomment-1227359748))**: Tested this in iOS 15.0 / iPad 9th generation Simulator and the UI language of the Keyman has been changed to Amharic Successfully.
bharanidharanj commented 2 years ago

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPHONE_PORTRAIT: portrait mode in iPhone

bharanidharanj commented 2 years ago

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone

bharanidharanj commented 2 years ago

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

GROUP_IPAD_PORTRAIT: portrait mode in iPad

GROUP_IPAD_LANDSCAPE: landscape mode in iPad

bharanidharanj commented 2 years ago

SUITE_SYSTEM_KEYBOARD:

GROUP_IPHONE_PORTRAIT: portrait mode in iPhone

GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone

bharanidharanj commented 2 years ago

SUITE_SYSTEM_KEYBOARD:

GROUP_IPAD_PORTRAIT: portrait mode in iPad

GROUP_IPAD_LANDSCAPE: landscape mode in iPad

bharanidharanj commented 2 years ago

SUITE_SWITCH_LOCALE:

GROUP_IPHONE_PORTRAIT: portrait mode in iPhone

GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone

bharanidharanj commented 2 years ago

SUITE_SWITCH_LOCALE:

GROUP_IPAD_PORTRAIT: portrait mode in iPad

GROUP_IPAD_LANDSCAPE: landscape mode in iPad