[web] 13.0 Pre-beta checklist

[mcdurdin]

Keyman for Web Acceptance Test Procedures

These test procedures are to be run before any beta or stable build is published. Copy these checklists of tests before PRs are merged into the beta and stable branches.

Remember that KeymanWeb is embedded into the iOS and Android apps, thus we need to ensure that those work properly with a KMW release as well. See the Android and iOS instructions.

Visual Inspection of master branch

• [x] Verify latest CI build of the KeymanWeb master is successful
• [x] Verify latest CI build of the Android master is successful
• [x] Verify latest CI build of the iOS master is successful
• [x] Verify web/history.md contains information on all the current changes.

Testing Platforms

Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:

• [x] Windows / Internet Explorer (v11+, as earlier versions do not support MutationObservers)
  • [ ] Testing with v10 may be worth it for the second and third page below; the first page will not be supported.
• [ ] Windows / Edge
• [x] Windows / Firefox
• [x] Windows / Chrome
• [x] MacOS / Safari
• [x] MacOS / Chrome
• [x] MacOS / Firefox
• [ ] Linux / *
• [ ] Android / native browser (without external keyboard)
• [ ] Android / native browser (with external keyboard)
• [ ] Android / Chrome (without external keyboard)
• [ ] Android / Chrome (with external keyboard)
• [x] iPhone / Safari (without external keyboard)
• [x] iPhone / Safari (with external keyboard)
• [x] iPad / Safari (without external keyboard)
• [x] iPad / Safari (with external keyboard)

Tests

• [ ] Utilize the web/testing/attachment-api samples page to ensure the following:

  • [ ] 'auto' attachment mode properly and automatically attaches to each dynamically-added control.
  • [ ] 'manual' attachment mode results in a page with no KMW-activated controls.
  • [ ] Each 'attached' control functions as expected for the keyboard(s) selected.
  • [ ] KeymanWeb properly attaches to and detaches from each relevant type of control with the page's API-interface controls.
  • [ ] Confirm that the independent-keyboard management API functions correctly.
  • While we do have automated tests as a first layer of defense for Attachment API issues, the behavior and interactions are complex enough to warrant manual inspection as well.

• [ ] Utilize the web/samples/minified.html sample page to ensure the following:

  • [ ] KMW properly handles input to the controls.
  • [ ] Attempt to add the following keyboards:
  • [ ] By keyboard name: "sil_ipa". (For our currently-deprecated IPA keyboard.)
    • [ ] For physical devices, ensure the key sequence n> produces ŋ.
    • [ ] For touch devices, ŋ should be a subkey of n - ensure it works.
  • [ ] By language code: "km".
    • Note that if "km" doesn't return the khmer_angkor keyboard, you'll want to load that one by keyboard name for these tests.
    • [ ] If using a physical keyboard, ensure the key sequence CTRL ALT B produces ឞ.
    • [ ] Type in the following sequence on touch: ស, ុ, ្រ (subkey of រ), ក. You should see ស្រុក.
    • [ ] On a physical device, typing the following (US) keys in sequence performs the same test: s, u, j, r, k.
    • [ ] Continuing from the last test, hit backspace in sequence. As this is a reorder (keyboard) rule test, you should see:
    • ស្រុ
    • ស្រ
    • ស
  • [ ] By language name: "spanish".
  • [ ] Swap to the Swedish keyboard.
  • [ ] If using a physical keyboard, ensure the key sequence 'e produces é and that o\e produces œ.
  • [ ] If using a touch-based keyboard, use long-press . to output ' and then press e. The two characters should not combine.
  • [ ] If using a touch-based keyboard, ensure that long-press p and SHIFT + long-press g displays properly and that the subkeys produce the expected output.
  • [ ] If using a touch-based keyboard, temporarily edit the page to exclude attachType: 'auto' from the initialization function's parameter object and ensure that KMW no longer activates for any controls on the page.
  • [ ] If using a touch-based keyboard, add the following keyboard by name: "sil_cameroon_qwerty"
  • [ ] Ensure the top two rows of the symbol layer work correctly. (These have unusual 'layer' settings.)
  • [ ] Ensure that the tri-color key (on both 'default' and 'shift' layers) changes layer appropriately and that keys pressed automatically transition back to the default layer. (The 'default' layer's variant is very unusual, as it matches a keyboard rule and changes layer, but doesn't generate direct output!)
  • [ ] Test that a subkey on the 'default' and on the 'shift' layers output the expected character.
  • Automated tests now provide a first layer of defense against all of the key sequence output checks.
  • It may be possible to additionally automate checks for keyboard requests, though this could destabilize the tests if issues among the test machine, the BrowserStack-based test server, and the Keyman server.

• [ ] Utilize the web/testing/keyboard-errors/ page to ensure the following:

  • [ ] Select each 'debugging' keyboard and ensure the appropriate error is displayed.
  • [ ] Ensure that both the "wrong-filename" and "non-parsable" keyboards display immediate errors.
  • This latter point should be possible to automate without too much issue in the future.

[jahorton]

Things of note (from Safari - macOS and Chrome-emulated Safari - iPhone):

• [ ] In prior versions, "add[ing] a keyboard by BCP-47 language code" used to return different keyboards.
  • For example, km used to return the khmer_angkor keyboard. It now retrieves the NiDA one (basic_kbdkni) by default.
  • Also of note: und-fonipa returns the deprecated ipauni11 keyboard instead of sil_ipa.

Noted in Windows tests:

• [x] Attempting to use BKSP within a design-mode