ni / nimble

The NI Nimble Design System
https://nimble.ni.dev
MIT License
29 stars 9 forks source link

Run webkit tests on pipeline builds #2237

Open m-akinc opened 4 days ago

m-akinc commented 4 days ago

Pull Request

🤨 Rationale

Closes #990

👩‍💻 Implementation

If we try to run the webkit tests on the CI as-is, a couple things can happen:

  1. The webkit browser sticks around after it has finished running tests and interferes with subsequent test execution. It connects to the karma server and starts running tests that were meant for chrome or firefox.
  2. The npm run validate step of the CI hangs. Presumably, the webkit browser process is keeping everything from exiting properly.

We can avoid those problems by killing the webkit browser process (MiniBrowser) after it finishes. To do this, we pull the source for karma-webkit-launcher into our repo and make very minor changes (which are documented in the source file). We can identify the MiniBrowser process that needs killing (multiple can run concurrently) because it has been orphaned by the bash process that launched it. The following is an abridged example of the output of ps at the point where the launcher is cleaning up:

 PID    PPID COMMAND
7673    7671 npm run test-concurrent (nimble-components)
7716    7673 sh
7729    7716 node
7771    7729 sh
7773    7771 npm run test-chrome:verbose (nimble-components)
7779    7729 sh
7782    7779 npm run test-firefox:verbose (nimble-components)
7783    7729 sh
7788    7783 npm run test-webkit:verbose (nimble-components)
7860    7788 sh
7863    7860 node
8259    7863 bash
8274    8259 MiniBrowser  <------------------ live, active instance
8593    8274 WPENetworkProce
8601    8274 WPEWebProcess

7672    7668 npm run test
7997    7672 sh
8602    7997 npm run test-webkit:verbose (spright-components)
8648    8602 sh
8649    8648 node
8680    8649 bash <defunct> <--------------- defunct/zombie bash
8686       1 MiniBrowser    <------------ orphaned webkit browser
8694    8686 WPENetworkProce
8697    8686 WPEWebProcess

🧪 Testing

Build runs without hanging. The validation step still takes roughly 6 minutes, even though it is running most of the nimble-components tests an additional time.

[test-concurrent:nimble-components] [test-webkit:verbose] Safari 17.0 (Linux x86_64): Executed 4595 of 4646 (skipped 51) SUCCESS (5 mins 11.218 secs / 4 mins 17.806 secs)

✅ Checklist