keymanapp / keyman

Keyman cross platform input methods system running on Android, iOS, Linux, macOS, Windows and mobile and desktop web

https://keyman.com/

Other

403 stars 112 forks source link

fix(developer): reconnect `--full-test` in kmcmplib build and enable for CI #12631

Closed mcdurdin closed 1 week ago

mcdurdin commented 2 weeks ago

The --full-test parameter in kmcmplib build.sh has not been working. This PR reconnects the parameter and also enables it by default for CI test builds (not release builds).

Note: the following keyboards are excluded because the snapshot versions have issues that the compiler now flags (these issues have all been resolved):

(#12623) masaram_gondi: due to #11806
(#12631) anii, sil_kmhmu: as ico references have mismatching case
(#12631) fv_statimcets, fv_nuucaanul: as these include U+2002 which is not treated as whitespace on mac arch

See developer/src/kmcmplib/tests/get-test-source.sh

These keyboards could be included again if we update the snapshot to a later version -- but that requires a rebuild of the fixture expected keyboards with v16.0 compiler, so will necessarily exclude other keyboards which are now v17+.

Fixes: #12623

@keymanapp-test-bot skip

keymanapp-test-bot[bot] commented 2 weeks ago

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

Android
Developer
iOS
- Keyman for iOS (simulator image)
- FirstVoices Keyboards for iOS (simulator image)
- TestFlight internal PR build version - 18.0.136 (0.12631.12235)
Keyboards
- Test Keyboards
Linux
macOS
- Keyman for macOS
Web
- KeymanWeb Test Home
Windows

mcdurdin commented 2 weeks ago

Linux wasm build failed with:

 223/701 anii                                    FAIL            0.08s   exit status 108
 636/701 sil_kmhmu                               FAIL            0.08s   exit status 108
 685/701 naijatype                               FAIL            0.08s   exit status 108

mac wasm build failed with:

 59/701 naijatype                               FAIL            0.08s   exit status 108

anii error is due to mismatch in case in bitmap filename. Filename is anii.ico; reference is Anii.ico.
sil_kmhmu error is due to mismatch in case in bitmap filename. Filename is sIL_kmhmu.ico; reference is sil_kmhmu.ico.
naijatype error is due to backslashes in bitmap filename. Resolved in 3c680e3

mac arch build failed with:

121/701 fv_statimcets    FAIL    exit status 108
> fv_statimcets.kmn(213): SevError | 0x00a
128/701 fv_nuucaanul     FAIL    exit status 108
> fv_nuucaanul.kmn(147): SevError | 0x00a

0x00a is Invalid Token. Turns out that those lines both have a U+2002 at the end, which in WASM and VC++ builds is trimmed by u16trim() and friends (via iswspace()), but it seems in mac arch build, is not.

mcdurdin commented 2 weeks ago

Follow-up disabled tests in 19.0 in #12634.

mcdurdin commented 2 weeks ago

From Developer-Linux test:

06:09:27   693/697 xinaliq                                 OK              0.02s
06:09:27   694/697 api-test                                OK              0.02s
06:09:27   695/697 uset-api-test                           OK              0.01s
06:09:27   696/697 gtest-compiler-test                     OK              0.01s
06:09:27   697/697 gtest-km_u16-test                       OK              0.01s
06:09:27   
06:09:27   Ok:                 697
06:09:27   Expected Fail:      0
06:09:27   Fail:               0
06:09:27   Unexpected Pass:    0
06:09:27   Skipped:            0
06:09:27   Timeout:            0

And same for Windows, macOS logs. So looks like these tests are running correctly in CI now. And the 697 tests took 17 seconds in total to run on Linux, 45 seconds on Windows (on a slower agent), 14 seconds on macOS.

(This means also we can remove the disabled 'regression' CI test step because this result is pretty effective for regression testing.)

keyman-server commented 1 week ago

Changes in this pull request will be available for download in Keyman version 18.0.140-alpha