E2E interop test failures

webrtc / samples

WebRTC Web demos and samples

https://webrtc.github.io/samples

BSD 3-Clause "New" or "Revised" License

13.75k stars 5.69k forks source link

E2E interop test failures #1614

Closed fippo closed 10 months ago

fippo commented 11 months ago

The nightly interop tests started failing with https://github.com/webrtc/samples/actions/runs/5723012987

This looks like a mix of

chromedriver refusing to recognize the symlink in browsers/bin
Chromedriver 115 being incompatible with M117 after a certain version.

The path is easy to fix (tested here) but after that it is still running into timeout issues

fippo commented 10 months ago

The difference between the last known good run https://github.com/webrtc/samples/actions/runs/5711022621/job/15472020011 and the first bad one https://github.com/webrtc/samples/actions/runs/5723012987/job/15509825644#step:8:31 is that the latter shows

Selenium Manager binary found at 
    /home/runner/work/samples/samples/node_modules/selenium-webdriver/bin/linux/selenium-manager

fippo commented 10 months ago

@bonigarcia you might know since I saw your twe..whatever one calls it now :-)

It seems that upgrading from 4.10.0 to 4.11.0 broke the custom path we use (together with the very old travis-multirunner) via setChromeBinaryPath. Does that ring a bell?

bonigarcia commented 10 months ago

It might be a bug on Selenium. Can you please open a bug report here?

I am reporting this in the Selenium TLC Slack channel as well.

fippo commented 10 months ago

Already fixed, just needs to be released before unpinning this. Thank you @bonigarcia!