search

browserstack / github-actions

A collection of GitHub Actions for BrowserStack. For internal reference: https://browserstack.atlassian.net/wiki/spaces/ENG/pages/1750928573/GitHub+Integration+via+GitHub+Actions
MIT License
54 stars 36 forks source link

Parallel jobs seems to misuse tunnels #11

Open SevenOutman opened 3 years ago

SevenOutman commented 3 years ago

I'm using karma-browserstack-launcher in GitHub Actions to run tests on several browsers.

strategy:
  matrix:
    browser: ['bs_win_ie11', 'bs_win_edge', 'bs_mac_chrome', 'bs_mac_firefox', 'bs_mac_safari']

The YAML is creating 5 parallel jobs, each running tests on corresponding browser. But the jobs' logs shows that the parallel jobs are receiving messages from each other's tunnel.

The bs_win_ie11 job (the first to run):

29 05 2021 11:16:10.261:INFO [karma-server]: Karma v6.3.2 server started at http://localhost:9876/
29 05 2021 11:16:10.264:INFO [launcher]: Launching browsers bs_win_ie11 with concurrency unlimited
29 05 2021 11:16:10.328:INFO [launcher]: Starting browser IE 11.0 (Windows 10) on BrowserStack
29 05 2021 11:16:14.694:INFO [launcher.browserstack]: IE 11.0 (Windows 10) session at https://automate.browserstack.com/builds/f04621633b17d4e093614166b355b0fd76eff56e/sessions/b1cff1ace638252d23c60c66317a879569443321
29 05 2021 11:16:15.600:WARN [web-server]: 404: /bac.jpg
29 05 2021 11:16:15.625:WARN [web-server]: 404: /elva-fairy-480w.jpg
29 05 2021 11:16:16.500:INFO [Chrome 90.0.4430.72 (Mac OS 10.15.7)]: Connected on socket xoWxToYEtv0CKgzMAAAB with id 87431852

The last line seems to indicate that bs_win_ie11 is receiving messages from a Chrome session. And the test output that follows also came from Chrome instead of IE.

The bs_mac_chrome job:

29 05 2021 11:15:38.179:INFO [karma-server]: Karma v6.3.2 server started at http://localhost:9876/
29 05 2021 11:15:38.183:INFO [launcher]: Launching browsers bs_mac_chrome with concurrency unlimited
29 05 2021 11:15:38.246:INFO [launcher]: Starting browser Chrome latest (OS X Big Sur) on BrowserStack
29 05 2021 11:15:42.538:INFO [launcher.browserstack]: Chrome latest (OS X Big Sur) session at https://automate.browserstack.com/builds/f04621633b17d4e093614166b355b0fd76eff56e/sessions/18b1facd3c4004febe69c1f941a1890c02f0f717
29 05 2021 11:20:50.213:WARN [launcher.browserstack]: Chrome latest (OS X Big Sur) has not captured in 300000 ms, killing.

Seems that bs_mac_chrome session is occupied by the bs_win_ie11 job thus the bs_mac_chrome failed due to timeout.

My workaround for now is to run these jobs sequentially to avoid this problem, but it's not a practical solution because it takes too long to finish all sequential jobs.

FYI I'm using a sponsored plan on BrowserStack which have up to 5 parallel running threads.