search

jitsi / jitsi-meet-torture

Apache License 2.0
161 stars 149 forks source link

Selenium setup #633

Closed alexander-potemkin closed 7 months ago

alexander-potemkin commented 7 months ago

Could you please, share, the selenium setup that is required?

I've tried docker-compose with selenium-3* version, but it's failing now, which is due to selenium-4 update in the source code, I guess.

I've also tried:

docker network create grid
docker run -d -p 4442-4444:4442-4444 --net grid --name selenium-hub selenium/hub:latest
docker run -d --net grid -e SE_EVENT_BUS_HOST=selenium-hub \
    --shm-size="2g" \
    -e SE_EVENT_BUS_PUBLISH_PORT=4442 \
    -e SE_EVENT_BUS_SUBSCRIBE_PORT=4443 \
    selenium/node-chrome:latest

as per the selenium doc, but it doesn't work well:

[14] No participant type specified for prefix: web.participant2, will use Chrome...
[14] TestFailure:
org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Host info: host: 'localhost', ip: '127.0.1.1'
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:620)
    at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:236)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:143)
    at org.jitsi.meet.test.web.WebParticipantFactory.startWebDriver(WebParticipantFactory.java:347)
    at org.jitsi.meet.test.web.WebParticipantFactory.doCreateParticipant(WebParticipantFactory.java:122)
    at org.jitsi.meet.test.web.WebParticipantFactory.doCreateParticipant(WebParticipantFactory.java:38)
    at org.jitsi.meet.test.base.ParticipantFactory.createParticipant(ParticipantFactory.java:35)
    at org.jitsi.meet.test.base.ParticipantHelper.createParticipant(ParticipantHelper.java:163)
    at org.jitsi.meet.test.web.WebTestBase.joinParticipant(WebTestBase.java:303)
    at org.jitsi.meet.test.web.WebTestBase.joinParticipantAndWait(WebTestBase.java:422)
    at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipantsInternal(WebTestBase.java:155)
    at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipants(WebTestBase.java:126)
    at org.jitsi.meet.test.AudioOnlyTest.setupClass(AudioOnlyTest.java:44)
    at org.jitsi.meet.test.base.AbstractBaseTest.setupClassPrivate(AbstractBaseTest.java:195)
<...>

Would you mind sharing your setup, please?

damencho commented 7 months ago

Doesn't these instructions work for you? https://github.com/jitsi/jitsi-meet-torture/tree/master/doc/grid

alexander-potemkin commented 7 months ago

@damencho , thanks for a prompt responce!

I have to confess, that I missed the doc folder there, my bad.

I'm still getting an error:

[14] TestFailure:
org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Host info: host: 'localhost', ip: '127.0.1.1'
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:620)
    at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:236)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:143)
    at org.jitsi.meet.test.web.WebParticipantFactory.startWebDriver(WebParticipantFactory.java:347)
    at org.jitsi.meet.test.web.WebParticipantFactory.doCreateParticipant(WebParticipantFactory.java:122)
    at org.jitsi.meet.test.web.WebParticipantFactory.doCreateParticipant(WebParticipantFactory.java:38)
    at org.jitsi.meet.test.base.ParticipantFactory.createParticipant(ParticipantFactory.java:35)
    at org.jitsi.meet.test.base.ParticipantHelper.createParticipant(ParticipantHelper.java:163)
    at org.jitsi.meet.test.web.WebTestBase.joinParticipant(WebTestBase.java:303)
    at org.jitsi.meet.test.web.WebTestBase.joinParticipantAndWait(WebTestBase.java:422)
    at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipantsInternal(WebTestBase.java:155)
    at org.jitsi.meet.test.web.WebTestBase.ensureTwoParticipants(WebTestBase.java:126)
    at org.jitsi.meet.test.AudioOnlyTest.setupClass(AudioOnlyTest.java:44)
    at org.jitsi.meet.test.base.AbstractBaseTest.setupClassPrivate(AbstractBaseTest.java:195)
<...>
alexander-potemkin commented 7 months ago

Just in case, I'm using:

./scripts/malleus.sh --conferences=1 --participants=4 --senders=1 --audio-senders=2 --duration=120 --room-name-prefix=hammertesting --hub-url=http://127.0.0.1:4444/wd/hub --instance-url=https://domain.com

And adjusted set of tests, that used to work with selenium-3.

alexander-potemkin commented 7 months ago

I can see, however, that mvn from the docs is failing all the same:

[ERROR] Tests run: 457, Failures: 1, Errors: 0, Skipped: 456, Time elapsed: 186.821 s <<< FAILURE! - in TestSuite
[ERROR] setupClassPrivate(org.jitsi.meet.test.UDPTest)  Time elapsed: 182.164 s  <<< FAILURE!
org.openqa.selenium.SessionNotCreatedException:
Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Host info: host: 'revizor', ip: '127.0.1.1'
Build info: version: '4.13.0', revision: 'ba948ece5b*'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-92-generic', java.version: '11.0.21'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {acceptInsecureCerts: false, browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*, allow-insecure-localhost, use-fake-ui-for-media-stream, use-fake-device-for-media-s..., disable-plugins, mute-audio, disable-infobars, autoplay-policy=no-user-ges..., auto-select-desktop-capture..., no-sandbox, disable-dev-shm-usage, disable-setuid-sandbox, headless, window-size=1400x600, disable-gpu, use-file-for-fake-audio-cap...], extensions: [], prefs: {intl.accept_languages: en-US}}, goog:loggingPrefs: org.openqa.selenium.logging...}]}]
Capabilities {acceptInsecureCerts: false, browserName: chrome, goog:chromeOptions: {args: [--remote-allow-origins=*, allow-insecure-localhost, use-fake-ui-for-media-stream, use-fake-device-for-media-s..., disable-plugins, mute-audio, disable-infobars, autoplay-policy=no-user-ges..., auto-select-desktop-capture..., no-sandbox, disable-dev-shm-usage, disable-setuid-sandbox, headless, window-size=1400x600, disable-gpu, use-file-for-fake-audio-cap...], extensions: [], prefs: {intl.accept_languages: en-US}}, goog:loggingPrefs: org.openqa.selenium.logging...}
    at org.jitsi.meet.test.UDPTest.setupClass(UDPTest.java:38)
Caused by: java.lang.RuntimeException: NettyHttpHandler request execution error
    at org.jitsi.meet.test.UDPTest.setupClass(UDPTest.java:38)
Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException: Request timeout to localhost/127.0.0.1:4444 after 180000 ms
    at org.jitsi.meet.test.UDPTest.setupClass(UDPTest.java:38)
Caused by: java.util.concurrent.TimeoutException: Request timeout to localhost/127.0.0.1:4444 after 180000 ms

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   UDPTest>AbstractBaseTest.setupClassPrivate:195->setupClass:38->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:422->WebTestBase.joinParticipant:303 » SessionNotCreated
alexander-potemkin commented 7 months ago

I can see that:

Driver info: driver.version: RemoteWebDriver
    at org.jitsi.meet.test.AudioOnlyTest.setupClass(AudioOnlyTest.java:44)
Caused by: java.util.concurrent.TimeoutException
    at org.jitsi.meet.test.AudioOnlyTest.setupClass(AudioOnlyTest.java:44)

If it times-out, is there a way to increase the time-out? I'm not running the tests on a top-notch hardware, so it might be the reason.

I tried to increase SE_NODE_SESSION_TIMEOUT=600 in node-docker, but the test is still failing in 3 minutes: Total time: 03:22 min - so, there must be some other timeout, probably.

Or, I'm looking in a completely wrong direction...

damencho commented 7 months ago

[ERROR] setupClassPrivate(org.jitsi.meet.test.UDPTest) Time elapsed: 182.164 s <<< FAILURE! org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. Host info: host: 'revizor', ip: '127.0.1.1'

If you have this one, fix this one first. Is this correctly opening http://127.0.0.1:4444

Not sure why the error reports different address: Host info: host: 'localhost', ip: '127.0.1.1'

damencho commented 7 months ago

Mind that one chrome needs few cores and at least 8GB of RAM

alexander-potemkin commented 7 months ago

Thank you! That was resources case, indeed. Appreciate your prompt help and hints 🙏🏻