gemini-testing / gemini

💀💀💀[DEPRECATED] Use hermione
https://github.com/gemini-testing/hermione
MIT License
1.5k stars 150 forks source link

Using Gemini with Chromedriver directly [without Selenium server], runs precisely '9' tests #822

Open satishautade opened 6 years ago

satishautade commented 6 years ago

I suspect if there is a maximum limit on the connections to the chromedriver somewhere. Shouldn't the same connection to chromedriver be reused for each test? Here is the test setup code Using gemini --version: 5.0.0-alpha.8 ...

.gemini.conf.js file:

module.exports = {
    rootUrl: 'http://0.0.0.0:3000',
    gridUrl: 'http://127.0.0.1:4444/wd/hub',
    windowSize: '1280x1024',
    compositeImage: true,
    screenshotsDir: './gemini/screens',
    system: {
        projectRoot: __dirname,
        ctx: {
            appPages: [ /* contains 60 page URLs */ ]
        },
        plugins: {
            'html-reporter': {
                enabled: true,
                path: 'test/gemini/reports',
                errorsOnly: false
            }
        }
    },
    browsers: {
        chrome: {
            desiredCapabilities: {
                browserName: 'chrome',
                chromeOptions: {
                    args: [ 'disable-gpu', 'headless', 'no-sandbox' ]
                }
            }
        }
    }
};

gemini-runner.js file:

const chromedriver = require('chromedriver');

startChromedriver();
startGemini();

function startChromedriver() {
    chromedriver.start( [
        '--url-base=wd/hub',
        '--port=4444',
        '--verbose'
    ] );
}

function startGemini() {
    const gemini = new Gemini( ".gemini.conf.js" );
        gemini.test( testFiles, {} )
        .finally( () => {
            chromedriver.stop();
        } );
}

Test source code:

gemini.suite( 'app-ui', () => {
    gemini.ctx.appPages.forEach( url => {
        gemini.suite( url, suite => {
            suite.setUrl( url )
                .setCaptureElements( 'body' )
                .capture( 'default', actions => actions.wait( 10 ) );
        } );
    } );
} );

Command used to run the test:

node gemini-runner.js

Result: This test suite takes array of about 60 page URLs (from ctx) and captures screenshots for each page. If we just use the chromedriver then first 9 URLs are hit and tests are run for them. But anything from 10th page URL starts failing as it is unable to connect to chromedriver with following error

Error: connect ECONNREFUSED 127.0.0.1:4444
    at Object.exports._errnoException (util.js:1020:11)
    at exports._exceptionWithHostPort (util.js:1043:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1090:14)

    GeminiError: Unable to connect to http://127.0.0.1:4444/wd/hub.
    at GeminiError (/app/node_modules/gemini/lib/errors/gemini-error.js:5:9)
    at initSession.then.then.then.then.then.catch (/app/node_modules/gemini/lib/browser/new-browser.js:112:43)
    at tryCatcher (/app/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/app/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/app/node_modules/bluebird/js/release/promise.js:689:18)
    at Async._drainQueue (/app/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/app/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/app/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

...

Expected behaviour: It should run through all the page URLs like it does for first 9.

...

vvscode commented 6 years ago

any updates to the topic?

sipayRT commented 6 years ago

hi but it's no gemini problem but chromedriver that limit your maximum connect count and reject connection

vvscode commented 6 years ago

@sipayRT is there a way to limit number of simultaneous connections ?

mobz commented 6 years ago

@sipayRT The tests are being run in series, not in parallel, so we expect the available connections to be reused - perhaps it is a problem closing the connections somewhere?

sipayRT commented 6 years ago

you can use suitesPerSession and sessionsPerBrowser options to configure working with browser sessions. See docs about it - https://gemini-testing.github.io/doc/config.html

jbrosi commented 6 years ago

Had the same problem. Solved it by running chromedriver via process.execFile:

import chromedriver from 'chromedriver'
const args = ['--port=4444', '--url-base=wd/hub']
chromeDriverChildProcess = childProcess.execFile(chromedriver.path, args, () => {
  //...
})

Stopping via chromeDriverChildProcess.kill()