fix(web): ios popup positioning and style

[mcdurdin]

Fixes #5929. Fixes #6381.

This fixes four separate issues with the longpress popups and key previews:

1. The key preview was offset by a couple of pixels (unreported issue).
2. The background 'shim' did not cover the key caps, which caused them to look startlingly white when the rest of the background greyed down (#5929).
3. The popup menu was incorrectly wrapping to two rows due to sizing model mismatches (#6381).
4. The popup menu would be realigned incorrectly for the in-app keyboard, when moving it to keep it in the keyboard area (#5929).

Longpress iOS (Chrome simulation):

Key preview iOS (Chrome simulation):

User Testing

• We need to verify the behaviour of the longpress menu and the key preview. If possible, capture screenshots of both of these elements when running the tests, so we can compare.

• Suggest using sil_euro_latin keyboard for this test, as it has many big longpress menus.

• Test single-row long presses, such as long-press m, and multi-row, such as long-press g or e.

TEST_IOS: Verify that the in-app keyboard and system keyboard both display longpress menus and key previews correctly. Note that the longpress menu will be constrained to fit within the keyboard area on iOS, due to limitations with iOS system keyboards. Please test on iPhone 13 Pro and iPhone SE. Consider other devices also. Please test with 4 and 5 row keyboards. TEST_ANDROID: Verify that the in-app keyboard and system keyboard both display longpress menus and key previews correctly. TEST_WEB_IOS: Verify that the web keyboard in unminified/testing, on an iOS simulator or device, displays longpress menus and key previews correctly. Please test on iPhone 13 Pro and iPhone SE. Consider other devices also. Please test with 4 and 5 row keyboards. TEST_WEB_ANDROID: Verify that the web keyboard in unminified/testing, on an Android emulator or device, displays longpress menus and key previews correctly.

[keymanapp-test-bot[bot]]

User Test Results

Test specification and instructions

• ✅ TEST_IOS (PASSED): ): I am comfortable with going ahead with that very minor artifact. Thank you for your attention to detail :grin:
• ✅ TEST_ANDROID (PASSED): Round 2 passed on Pixel 2 emulator with Android API 29 (notes)
• ✅ TEST_WEB_IOS (PASSED): Tested this in Unminified page on iPhone 13 Pro Simulator and it seems to be working as expected. (notes)
• ✅ TEST_WEB_ANDROID (PASSED): Re-tested this on Android emulator (API 30) in Unminified Page and it seems to be working as expected. (notes)

Test Artifacts

• KeymanWeb Test Home
• Keyman for Android apk
• FirstVoices Keyboards for Android apk
• KMSample1 apk
• KMSample2 apk
• Keyman for iOS (simulator image)
• FirstVoices Keyboards for iOS (simulator image)
• Keyman for Windows
• Keyman Developer
• Test Keyboards
• FirstVoices Keyboards for Windows

[darcywong00]

Test Results

• TEST_ANDROID (PASSED): Popup and longpress keys appear normal Tested on Pixel 2 emulator with Android API 29

In-app popup key

In-app longpress keys

System popup key

System longpress keys

[bharanidharanj]

• TEST_IOS (PASSED): Tested on iOS 15.2 Simulator and it is working as expected.

In-App Key Preview

System Keyboard Key Preview

In-App Key Long Press (single row)

In-App Key Long Press (multiple rows)

System Keyboard Long Press (single row)

System Keyboard Long Press (multiple rows)

[bharanidharanj]

• TEST_WEB_ANDROID (PASSED): Tested this on Android Emulator (API 29 - Version 10.0) and it is working as expected.

Key Preview - Unminfied Page

Long Press Single Row - Unminified Page

Long Press Multiple Rows - Unminified Page

[bharanidharanj]

• TEST_WEB_IOS (PASSED): Tested this on iOS 15.2 Simulator in the Keyman Web page and it is working as expected.

Key Preview - Unminified Page

Long Press - Single row -Unminified Page

Long Press - Multiple rows - Unminified Page

[mcdurdin]

Hmm, the positioning is certainly better than before. But it doesn't look quite right on KeymanWeb in iOS and Keyman for iOS. So will have another round.

Note that the problem is that Safari's CSS is diverging from Chrome here. I haven't identified what is triggering the difference yet.

Key Preview, Keyman for iOS

Longpress, Keyman for iOS

Key Preview, KeymanWeb on Safari / iOS

Longpress, KeymanWeb on Safari / iOS

Final issue: some keys are misaligned, only in Safari, not in Chrome:

[mcdurdin]

So I've done another round of tweaks to the positioning to try and get it working better. I have tested locally on my test build with Chrome on Windows (emulation and in developer frame), and in Safari on iOS, but not yet in iOS app.

But this sadly means retesting everything, so sorry @bharanidharanj and @darcywong00! The screenshots were very helpful, please keep them coming.

@keymanapp-test-bot retest all

[darcywong00]

• TEST_ANDROID (PASSED): Round 2 passed on Pixel 2 emulator with Android API 29

inapp keyboard - popup

inapp keyboard - longpress

system keyboard - popup

system keyboard - longpress

[bharanidharanj]

• TEST_WEB_ANDROID (PASSED): Re-tested this on Android emulator (API 30) in Unminified Page and it seems to be working as expected.

Unminified Page - Popup

Unminified Page - Long Press

[bharanidharanj]

• TEST_IOS (PASSED): Tested this on iPhone 13 Pro (iOS 15.2) Simulator and the long press functionality seems to be deviated from the alignment. I don't know whether my assumption is right or wrong.

In-App - Popup

In-App - Long Press

System Keyboard - Popup

System Keyboard - Long Press

[bharanidharanj]

• TEST_WEB_IOS (PASSED): Tested this on iPhone 13 Pro Simulator (iOS 15.2) in Unminified Page and it seems to be working as expected.

System Keyboard - Popup

System Keyboard - Long Press (Single row)

System Keyboard - Long Press (Multiple row)

[mcdurdin]

• TEST_IOS (PASSED): Tested this on iPhone 13 Pro (iOS 15.2) Simulator and the long press functionality seems to be deviated from the alignment. I don't know whether my assumption is right or wrong.

Yes, this doesn't look right... 😢

TEST_IOS FAILED

[mcdurdin]

Okay, I've fixed up the issues with the callout positioning on iOS.

@keymanapp-test-bot retest TEST_IOS

[MakaraSok]

TEST_IOS (FAILED): the alignment on the key when long pressed looks better, but the visible border between the base key and the subkey is still seen on the first row, i.e. key q, w, r, t, y and p.

in both in-app and system

The visible border between the preview key and its base key is seen on all rows. The preview key doesn't seen to be fully aligned center. The right part of the key looks longer than that of the left. 1st row

2nd row

3 row

The visible border between the subkey and its base is seen on the 1st row only, but not the 2nd and 3rd. No alignment issue seen though, AFA I can tell. 1st row

2nd row

3rd row

---

Bonus screen recording if they help in anyway.

in-app

https://user-images.githubusercontent.com/28331388/159399066-a35bad9a-4ff5-4255-94eb-9828dda5ad73.mov

system

https://user-images.githubusercontent.com/28331388/159399095-4aa55f5e-121c-44c8-a810-057567c38c01.mov

[mcdurdin]

Okay, I've tweaked the positions slightly with the latest commit. Can you check these again per spec?

Sample key preview from Safari on iOS (zoomed):

@keymanapp-test-bot retest TEST_IOS TEST_WEB_IOS

[bharanidharanj]

• TEST_IOS (FAILED): Re-tested this in iPhone 13 Pro (iOS 15.2) Simulator with the new PR (iOS) and I noticed that while long pressing the top row buttons the base key goes behind the multiple rows view. (eg., long pressing e letter would open the multiple rows popup, but the base key e is not shown)

In-App - Popup:

In-App - LongPress (Single row):

In-App LongPress (Multiple rows):

System Keyboard - popup:

System Keyboard - Long Press (Single row):

System Keyboard - Long Press (Multiple rows):

[bharanidharanj]

• TEST_WEB_IOS (PASSED): Tested this in Unminified page on iPhone 13 Pro Simulator and it seems to be working fine. Note: It is also working fine even in the Caps Lock is one.

Unminified Page : Popup

Unminified Page : Long Press (Single row)

Unminified Page : Long Press (Multiple rows)

[mcdurdin]

Please note that the longpress menu overlapping the e key is not a bug, but forced on us because iOS does not allow our longpress menu to be outside the keyboard frame.

But I'm still seeing some artifacts and alignment issues on your screenshots @bharanidharanj. I've highlighted a few of them from TEST_IOS; the TEST_WEB_IOS screenshots show the same issues. Neither the key preview nor the longpress are perfect.'

I tested on the iPhone SE 3rd Generation 15.4 Simulator and things looked pretty much pixel perfect there. However @bharanidharanj tested on iPhone 13 Pro and here things did not look right. So now to find out why.

Some examples:

1. antialiasing lines in junction between key tip and key cap (In-App - Popup, m key, Eurolatin):

   

2. key tip has wrong bottom alignment (In-App - Longpress, m key, Eurolatin):

   

3. key tip has wrong bottom alignment (In-App - Longpress, e key, Eurolatin).:

   

   (Same issues apply for o key, Long Press, System Keyboard)

4. key cap does not correctly overlap the key tip (System Keyboard - popup, p key, Eurolatin):

   

   A secondary issue here is that the key preview should probably be moved right by the border width?

5. key tip has wrong bottom alignment (System Keyboard - Longpress, m key, Eurolatin):

   

6. key tip does not overlap the longpress menu (web, Longpress, m key, Eurolatin):

   

• TEST_IOS: FAILED due to artifacts described above
• TEST_WEB_IOS: FAILED due to artifacts described above

[mcdurdin]

Okay, I've tried again and tested locally on both iPhone SE 3rd gen and iPhone 13 Pro to cater to the different resolutions. We should probably test against both those devices and with more than one keyboard -- the subpixel calculations can mask issues on different resolutions.

@keymanapp-test-bot retest TEST_IOS TEST_WEB_IOS

[bharanidharanj]

Okay, I've tried again and tested locally on both iPhone SE 3rd gen and iPhone 13 Pro to cater to the different resolutions. We should probably test against both those devices and with more than one keyboard -- the subpixel calculations can mask issues on different resolutions.

@keymanapp-test-bot retest TEST_IOS TEST_WEB_IOS

@mcdurdin Okay.

[bharanidharanj]

• TEST_IOS (FAILED): Re-tested this in iPhone 13 Pro (iOS 15.2) Simulator with the updated PR (iOS) and I noticed that while long pressing the top row buttons the key tip has wrong bottom alignment as mentioned by Marc. (in his previous comments)

In-App - Popup:

In-App - LongPress (Single row):

In-App - LongPress (Multiple rows):

System Keyboard - popup:

System Keyboard - Long Press (Single row):

System Keyboard - Long Press (Multiple rows):

[bharanidharanj]

• TEST_WEB_IOS (PASSED): Tested this in Unminified page on iPhone 13 Pro Simulator and it seems to be working as expected.

Unminified Page : Popup

Unminified Page : Single row

Unminified Page : Multiple rows

[mcdurdin]

• TEST_IOS (FAILED): Re-tested this in iPhone 13 Pro (iOS 15.2) Simulator with the updated PR (iOS) and I noticed that while long pressing the top row buttons the key tip has wrong bottom alignment as mentioned by Marc.

Good catch! I missed that when I was re-testing. Okay, I have put in a fix for that, can you retest please?

@keymanapp-test-bot retest TEST_IOS

[bharanidharanj]

• TEST_IOS (FAILED): Re-tested this in iPhone 13 Pro (iOS 15.2) Simulator with the updated PR (iOS) and I noticed that while long pressing the top row buttons the key tip has wrong bottom alignment as mentioned by Marc.

Good catch! I missed that when I was re-testing. Okay, I have put in a fix for that, can you retest please?

@keymanapp-test-bot retest TEST_IOS

@mcdurdin Okay. I will re-test it. Thanks.

[bharanidharanj]

• TEST_IOS (FAILED): Retested this with the attached PR build (Keyman 15.0.211-beta-6383) in iPhone SE and the key tip has wrong bottom alignment in Longpress (Multiple rows). I am seeing the same result in iPhone 13 Pro Simulator too.

In-App (Popup)

In-App - Long Press (Single row)

In-App - Long Press (Multiple rows)

System Keyboard - Popup

System Keyboard - Long Press (Single row)

System Keyboard - Long Press (Multiple rows)

[mcdurdin]

• TEST_IOS (FAILED): Retested this with the attached PR build (Keyman 15.0.211-beta-6383) in iPhone SE and the key tip has wrong bottom alignment in Longpress (Multiple rows). I am seeing the same result in iPhone 13 Pro Simulator too.

TEST_IOS (PASSED): I am comfortable with going ahead with that very minor artifact. Thank you for your attention to detail :grin:

[keyman-server]

Changes in this pull request will be available for download in Keyman version 15.0.223-beta