Regression Tests - Keyman for Mac

[MakaraSok]

Keyman for Mac Acceptance Test Procedures

1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
2. Copy these lists of tests into a new issue (for tier transitions) or a comment on the PR.
3. Use User Testing format for documenting tests.

User Testing

Setup Steps

1. Ensure the device is running the latest version of macOS (beta build for beta phase, alpha build for alpha phase
2. Uninstall previous version of Keyman on the device
3. Install latest master build (or whatever version is to be tested) of Keyman on the device

SUITE_BASIC_SMOKE_TEST: Basic smoke test

• TEST_VERSION: Verify that the version displayed in the About box is correct.
• TEST_DOWNLOAD_WIFI_OFF:
  • On device, turn Wifi off and/or disconnect Ethernet cable
  • Activate Keyman and select Keyman Configuration from the input methods menu. Click Download keyboard
  • Verify error message "Could not reach Keyman server!" is displayed (This is currently not happening. Bug reported as: https://github.com/keymanapp/keyman/issues/302)
  • Close Keyman configuration windows.
• TEST_DOWNLOAD_WIFI_ON:
  • On device, re-enable Internet connectivity
  • Activate Keyman and select Keyman Configuration from the input methods menu. Click Download keyboard.
  • Verify that page displays correctly (should have a Keyboard Search box with the prompt: Enter language or keyboard.
  • Search for Amharic. Should see several keyboards, including some by The Ge'ez Frontier Foundation and some by SIL.
  • Double-click any KMP file (in Finder). Should see a “Do you want the Keyman to install the keyboard from the file 'xxx.kmp'?“ message. If you click Install, the package should be installed (or updated) and the configuration window should display, showing that the keyboard(s) in the package are installed and enabled. If you click Cancel, the configuration window should not be displayed (but there is a known issue that the very first time after Keyman loads, if the configuration window has not been shown, it will be displayed even if the user clicks Cancel).

---

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

• GROUP_CHROME_URL_BAR: Chrome browser URL bar

• GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

• GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online Note: Not using the Office Extension for Chrome - not sure if that would make a difference

  • Beginning of paragraph (currently known compatibility issues steps 1, 3, 4 & 5)
  • Elsewhere in paragraph

• GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

• GROUP_SAFARI_URL_BAR: Safari browser URL bar

• GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

• GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online a) Beginning of paragraph (currently known compatibility issues steps 1, 3, 4 & 5, 6&7) b) Elsewhere in paragraph (occasionally steps 4 & 5 deletes a preceding (space?) character, but I haven't figured out when/why. Step 6 is not relevant. Step 7 fails for the same reason it doesn't work at the beginning of the paragraph. There seems to be a certain level of general twitchiness, with extra characters sometimes being displayed - sometimes only briefly - and characters being deleted.)

• GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

• GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

• GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online a) Beginning of paragraph b) Elsewhere in paragraph

• GROUP_MESSAGES_APP: Messages app a) New message with nobody filled in in To: line b) New message in existing conversation, such that the message area has grayed out text: "iMessage")

• GROUP_NOTES_APP: Notes app

• GROUP_TEXTEDIT: TextEdit

• GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 6 & 7. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)

• GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

• GROUP_MAIL: Mail (message body)

Test Cases

Select the Amharic keyboard. For each of the following applications/contexts, test that

• TEST_AMHARIC: (1). typing ta produces ታ; (2). typing t, left-arrow, a produces አት; (3). typing tt, left-arrow, a produces ታት; (4). typing tt, then mouse-clicking between the two ታ characters, and then typing a produces ታት.

  In each of the last three cases, the insertion point should end up before the final ት character.

• TEST_SHORTCUTS፡ (cont. from above) (5). common keyboard shortcuts for the app work as expected. For example:

  • ⌘-F (typically opens a Find dialog or control);
  • ⌘-O (File Open);
  • ⌘-X (Cut);
  • ⌘-C (Copy);
  • ⌘-V (Paste);
  • ⌘-A (Select All).

  (Note that in Google Docs, these commands correspond to both a Google-Docs command and a browser command. The "normal" behavior in all three browsers is to activate the Google Docs command, not the browser command, with the exception that ⌘-O in Safari opens Safari's standard macOS Open file dialog.)

  (6). with a few characters displayed, ⌘-A followed by typing t replaces selected characters with ት. (7). typing at, then ⌘-A, followed by a produces አ.

---

SUITE_OSK: OSK UI/Functionality Tests

• TEST_OSK_UI:
  • Verify behavior of "Always show on-screen keyboard" option
  • Verify behavior and appearance of On-Screen Keyboard and the menu that toggles it on and off.
  • With OSK visible, verify that switching keyboards updates the display to show the appropriate layout.
  • Verify that switching to system keyboard or other IM turns off Keyman OSK.

[keymanapp-test-bot[bot]]

User Test Results

Test specification and instructions

🟩 SUITE_BASIC_SMOKE_TEST: Basic smoke test

• ✅ TEST_VERSION (PASSED): 15.0.244-beta
• 🟩 TEST_DOWNLOAD_WIFI_OFF (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/302) is fixed.
• ✅ TEST_DOWNLOAD_WIFI_ON (PASSED): No issue found.

🟩 SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

• ✅ GROUP_CHROME_URL_BAR: Chrome browser URL bar

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: All work as expected. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: All work as expected.

• 🟩 GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

  • ✅ TEST_AMHARIC (PASSED): All work as expected.
  • 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6655) is fixed.

• ✅ GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found except the ones mentioned above. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found except the ones mentioned above.

• 🟩 GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online

  • ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
  • 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6656) is fixed.

• ✅ GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found except the ones mentioned above. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found except the ones mentioned above.

• ✅ GROUP_SAFARI_URL_BAR: Safari browser URL bar

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found.

• ✅ GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1122174675))**: No issue found.

• 🟩 GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

  • ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
  • 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6656) is fixed.

• 🟩 GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online

  • ✅ TEST_AMHARIC (PASSED): No issue found.
  • 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6657) is fixed.

• ✅ GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found.

• ✅ GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found.

• ✅ GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except the ones mentioned above. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except the ones mentioned above.

• ✅ GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found.

• ✅ GROUP_MESSAGES_APP: Messages app

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.

• ✅ GROUP_NOTES_APP: Notes app

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.

• ✅ GROUP_TEXTEDIT: TextEdit

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found.

• ✅ GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 6 & 7. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except the ones mentioned above. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except the ones mentioned above.

• ✅ GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found except the ones mentioned above. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found.

• ✅ GROUP_MAIL: Mail (message body)

  2 tests PASSED

  - ✅ **TEST_AMHARIC ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found. - ✅ **TEST_SHORTCUTS ([PASSED](https://github.com/keymanapp/keyman/issues/6436#issuecomment-1123118404))**: No issue found. (Command+O is N/A.)

🟩 SUITE_OSK: OSK UI/Functionality Tests

• 🟩 TEST_OSK_UI (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6578) is fixed.

[bharanidharanj]

SUITE_BASIC_SMOKE_TEST: Basic smoke test

• TEST_VERSION (PASSED): Successfully installed Keyman 16.0.4 build from Master branch. (in Mac OS 12.1)

  

• TEST_DOWNLOAD_WIFI_OFF (PASSED): Did not see any error message but it is showing blank window.

  

• TEST_DOWNLOAD_WIFI_ON (PASSED): Successfully downloaded and installed Amharic Keyboard.

  

[bharanidharanj]

GROUP_NOTES_APP: Notes app

• TEST_AMHARIC (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Notes App and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Notes App and it is working as expected.

GROUP_TEXTEDIT: TextEdit

• TEST_AMHARIC (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Text Editor and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Text Editor and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_URL_BAR: Chrome browser URL bar

• TEST_AMHARIC (PASSED): Tested this on Chrome browser in Mac 12.1 Monterey OS and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on Chrome browser in Mac 12.1 Monterey OS and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_URL_BAR: Safari browser URL bar

• TEST_AMHARIC (PASSED): Tested this in Safari (Mac Mini Version 12.1 Monterey) with the latest Keyman 16.0.4 build and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this in Safari (Mac Mini Version 12.1 Monterey) with the latest Keyman 16.0.4 build and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

• TEST_AMHARIC (PASSED): Tested this in FireFox Browser URL bar and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this in FireFox Browser URL bar and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

• TEST_AMHARIC (FAILED): Tested this on Atom - Snippet editor and it the compatibility issue remains for steps 3 and 4. Step 5 seems to be working fine with the current build. (16.0.4)

Compatibility issue in step 3 and 4:

• TEST_SHORTCUTS (PASSED): Tested this on Atom - Snippet editor and it seems to be working as expected. I did not see any compatibility issue

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_NOTES_APP: Notes app

• TEST_AMHARIC (PASSED): Tested this on Mac Mini OS, NOTES App and it seems to be working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on Mac Mini OS, NOTES App and all the commands are working as expected. (except command + O)I think this one is working only on Safari browser.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_TEXTEDIT: TextEdit

• TEST_AMHARIC (PASSED): Tested this with the latest Keyman 16.0.4 build and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this with the latest Keyman 16.0.4 build and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_MESSAGES_APP: Messages app

• TEST_AMHARIC (PASSED): Tested this on the Messages App with the latest Keyman 16.0.4 build and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on the Messages App with the latest Keyman 16.0.4 build and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• TEST_AMHARIC (PASSED): Tested in Chrome browser Google Document and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested in Chrome browser Google Document and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

• TEST_AMHARIC (PASSED): Tested this on Facebook Search bar and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on Facebook Search bar and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

• TEST_AMHARIC (PASSED): Tested this on Facebook Search Control bar and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on Facebook Search Control bar and it is working as expected.

GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

• TEST_AMHARIC (PASSED): Tested this on Facebook Search Control bar and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on Facebook Search Control bar and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

• TEST_AMHARIC (PASSED): Tested this in LibreOffice 7.3.1 and it is working fine, except the compatibility issues in Steps 3,4 and 5.
• TEST_SHORTCUTS (PASSED): Tested this LibreOffice 7.3.1 and the Shortcut keys are working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_MAIL: Mail (message body)

• TEST_AMHARIC (PASSED): Tested this on Mail (Message body) and it is working as expected.
• TEST_SHORTCUTS (PASSED): Tested this on Mail (Message body) and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 7 & 8. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)

• TEST_AMHARIC (PASSED): Tested this as per the acceptance test and it working as expected in the terminal.
• TEST_SHORTCUTS (PASSED): Tested this as per the acceptance test and it working as expected in the terminal.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

SUITE_OSK: OSK UI/Functionality Tests

• TEST_OSK_UI (PASSED): Tested Keyman On Screen Keyboard functionality, as per the test steps mentioned in the test suite and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online

• TEST_AMHARIC (FAILED): Tested this in Word Online (Office.com)in Chrome browser and it failed in 1, 3, 4, 6 and 7 Steps.
• TEST_SHORTCUTS (PASSED): Tested this in Word Online in Chrome browser and the Shortcut keys are working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online

• TEST_AMHARIC (PASSED): Tested this in Firefox_Word Online (from Office.com) and it seems to be working fine.
• TEST_SHORTCUTS (PASSED): Tested this in Firefox_Word Online (from Office.com) and it seems to be working fine.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online

• TEST_AMHARIC (PASSED): Tested this in Word Online document and it working fine except the known compatibility issues as it mentioned in the User Testing.
• TEST_SHORTCUTS (PASSED): Tested this in Word Online document and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• TEST_AMHARIC (PASSED): Tested this on Google document and it is working fine except the know compatibility issue for steps 3 and 4.
• TEST_SHORTCUTS (PASSED): Tested this on Google document and it is working as expected.

[bharanidharanj]

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• TEST_AMHARIC (PASSED): Tested this in Google online document and it is working fine except the known compatibility issue for steps 3,4 & 5.
• TEST_SHORTCUTS (PASSED): Tested this in Good Online document and it is working as expected.

[MakaraSok]

To be tested with the latest beta build

@keymanapp-test-bot retest all

[MakaraSok]

Test Results

Tested with Keyman 15.0.244-beta on macOS 12.3.1.

SUITE_BASIC_SMOKE_TEST: Basic smoke test

• TEST_VERSION (PASSED): 15.0.244-beta

• TEST_DOWNLOAD_WIFI_OFF (FAILED): a blank page instead of the error message is shown

  

• TEST_DOWNLOAD_WIFI_ON (PASSED): No issue found.

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_URL_BAR: Chrome browser URL bar

• TEST_AMHARIC (PASSED): All work as expected.
• TEST_SHORTCUTS (PASSED): All work as expected.

GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

• TEST_AMHARIC (PASSED): All work as expected.
• TEST_SHORTCUTS (FAILED): (7). outputs ታ instead of አ.

GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (FAILED): (6). and (7). output t and a instead of ት and አ.

GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_SAFARI_URL_BAR: Safari browser URL bar

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (FAILED): (6). and (7). output t and a instead of ት and አ.

GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (FAILED): (6). and (7). remove the selected text, but output nothing.

[MakaraSok]

Test Results

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_MESSAGES_APP: Messages app

• TEST_AMHARIC (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
• TEST_SHORTCUTS (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.

GROUP_NOTES_APP: Notes app

• TEST_AMHARIC (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
• TEST_SHORTCUTS (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.

GROUP_TEXTEDIT: TextEdit

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 6 & 7. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

• TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
• TEST_SHORTCUTS (PASSED): No issue found.

GROUP_MAIL: Mail (message body)

• TEST_AMHARIC (PASSED): No issue found.
• TEST_SHORTCUTS (PASSED): No issue found. (Command+O is N/A.)

SUITE_OSK: OSK UI/Functionality Tests

• TEST_OSK_UI (BLOCKED): Please clarify if the OSK should behave like that of the physical keyboard when typing/pressing on the key caps. For time being, they are different. Please see the table below.

keystrokes	physical keyboard	OSK
ta	ታ	ትአ

[MakaraSok]

SUITE_BASIC_SMOKE_TEST

• TEST_DOWNLOAD_WIFI_OFF (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/302) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_CHROME_FB_SEARCH_CONTROL

• TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6655) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_CHROME_WORD_ONLINE

• TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6656) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_SAFARI_GOOGLE_DOCS

• TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6656) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_SAFARI_WORD_ONLINE

• TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6657) is fixed.

[MakaraSok]

SUITE_OSK

• TEST_OSK_UI (SKIPPED): This test is to be included in the future test if this issue (https://github.com/keymanapp/keyman/issues/6578) is fixed.