codeceptjs / CodeceptJS

Supercharged End 2 End Testing Framework for NodeJS
http://codecept.io
MIT License
4.11k stars 723 forks source link

How to run some tests in Internet Explorer by using this command (npm run codeceptjs) #878

Closed ninpop25 closed 6 years ago

ninpop25 commented 6 years ago

I want to run some of my tests in Internet Explorer and I want to use this command to run (npm run codeceptjs) as my aim is to run tests and generate report with one command only.

On the other hand some of my other tests should run in Chrome

What do you get instead?

Provide console output if related. Use --verbose mode for more details.

# paste output here

Provide test source code if related

// paste test

Details

# paste suite config here
reubenmiller commented 6 years ago

This is an easy one (though a very verbose response).

Short answer

Set the browser setting in the CodeceptJS config to internet explorer instead of chrome.

Verbose anwser

If you want to CodeceptJS to test across different browsers, then you're only option is to use the WebDriverIO Helper as it communicates with a Selenium Server which then talks to the browser of your choice (assuming that you have the appropriate web driver. i.e. IEDriver.exe for Internet Explorer etc.). Though the Selenium Server setup is made easier by using selenium-standalone.

So assuming you already have your selenium-standalone server running, then you have to set the browser that you want to use for testing, the accepted values are "chrome", "firefox" and "internet explorer". Other browsers could be supported, though I haven't used them directly so check the selenium server docs for more information.

Here is an example of a basic configuration, which allows the user to override the url and browser by using environment variables (i.e. useful when using with Jenkins).

Codeceptjs config file

./codecept.conf.js

exports.config = {
  output: './output',
  helpers: {
    WebDriverIO: {
      url: process.env.URL || 'http://192.168.8.101',
      browser: process.env.BROWSER || 'chrome',
    },
    FileSystem: {},
  },
  mocha: {},
  bootstrap: false,
  teardown: null,
  hooks: [],
  tests: './tests/*_test.js',
  timeout: 10000,
  name: 'doler-js',
};

Running a test on Windows using a batch file.

rem Assuming you are in your root project folder.
cd "%~dp0"
set BROWSER=internet explorer
set URL=http://192.168.8.101
node "node_modules\codeceptjs\bin\codecept.js" run -c ./codecept.conf.js

Alternative windows wrapper for calling codeceptjs

I personally use a small batch wrapper script to handle calling CodeceptJS on windows, because this way you can still define your scripts in your package.json and it will work on Windows and MacOS.

codeceptjs.bat in root project folder

 @SETLOCAL
@SET PATHEXT=%PATHEXT:;.JS;=;%
echo Executing local codeceptjs (windows workaround)
node  "%~dp0\node_modules\codeceptjs\bin\codecept.js" %*

package.json

{
  "name": "doler-js",
  "version": "0.0.1",
  "description": "Tests",
  "license": "MIT",
  "shell": "C:\\Program Files\\Git\\git-bash.exe",
  "scripts": {
    "install:selenium": "npm install -g selenium-standalone@6.4.0 && selenium-standalone install",
    "start:codeceptjs": "./node_modules/.bin/codeceptjs",
    "start:server": "cmd /c start selenium-standalone start",
    "start:serverlinux": "selenium-standalone start",
    "start:phantom": "cmd /c start phantomjs --webdriver=4444 --ignore-ssl-errors=true",
    "shell": "codeceptjs shell",
    "test": "codeceptjs run -c ./codecept.conf.js --grep @framework --invert --steps --reporter mocha-multi",
    "test:withoutdocs": "codeceptjs run -c ./codecept.conf.js --grep \"@docs|@framework\" --invert --steps --reporter mocha-multi",
    "test:all": "codeceptjs run -c ./codecept.conf.js --steps --reporter mocha-multi"
  },
  "dependencies": {
    "chai": "^4.1.0",
    "chalk": "^1.1.3",
    "chokidar-cli": "^1.2.0",
    "co": "^4.6.0",
    "codeceptjs": "^1.1.1",
    "codeceptjs-webdriverio": "^1.0.1",
    "jquery": "^3.2.1",
    "js-yaml": "^3.10.0",
    "lodash": "^4.17.4",
    "mocha-junit-reporter": "^1.13.0",
    "mocha-multi": "^0.11.0",
    "mochawesome": "^2.2.0",
    "moment": "^2.18.1",
    "nconf": "^0.8.5",
    "nconf-yaml": "^1.0.2",
    "nightmare": "^2.10.0",
    "nightmare-iframe-manager": "rosshinkley/nightmare-iframe-manager",
    "nightmare-upload": "^0.1.1",
    "nightmare-xpath": "^2.0.2",
    "node-fetch": "^1.7.3",
    "present": "^1.0.0",
    "replace": "^0.3.0",
    "selenium-standalone": "^6.4.0",
    "shelljs": "^0.7.7",
    "shortid": "^2.2.8",
    "winston": "^2.4.0",
    "yamljs": "^0.3.0"
  },
  "devDependencies": {
    "eslint": "^4.13.1",
    "eslint-config-airbnb": "^16.1.0",
    "eslint-plugin-import": "^2.8.0",
    "eslint-plugin-jsx-a11y": "^6.0.3",
    "eslint-plugin-react": "^7.5.1"
  }
}

The run your project script as follows

npm run test:all

Warning

Don't just copy/paste the code because it probably won't work on your setup. I just wanted to give you an idea of how it could work/look like.

ninpop25 commented 6 years ago

@reubenmiller : Thanks for reply

I think I was not able to describe clearly what I was looking for,so I am doing so again.

suppose I have total of 100 tests , now out of this 100 I wan that some of tests say for example first 50 tests should Run in Chrome and left 50 tests that are need to be verified on IE should run on Internet Explore

I have this requirement of running some tests on IE from my client. below is my codecept.json

{ "output": "./output", "helpers": { "WebDriverIO": { "url": "https://formsviewertest.azurewebsites.net", "browser": "chrome", "windowSize": "maximize", "restart": false, "keepBrowserState": true, "keepCookies": true }, "TabSwitch": { "require": "./tabswitch_helper.js" } }, "include": { "I": "./steps_file.js", "loginPagePage": "./pages/loginPage.js", "manageCredentialPage": "./pages/manageCredential.js", "templatePage": "./pages/template.js", "uiHelper": "./pages/uiHelper.js", "fvAssert": "./pages/fvAssert.js", "listPage": "./pages/listPage.js", "fvData": "./InputData/login.js" }, "mocha": { "reporterOptions": { "codeceptjs-cli-reporter": { "stdout": "-", "options": { "verbose": false, "steps": true } }, "mocha-junit-reporter": { "stdout": "./output/console.log", "options": { "mochaFile": "./output/result.xml" } } } }, "bootstrap": "./config/start_server.js", "teardown": "./config/stopserver.js", "hooks": [], "tests": "**/fv*_test.js", "timeout": 10000, "name": "FV" }

reubenmiller commented 6 years ago

There are a few ways you can split the tests.

Method 1: Different Test folders

Separate the tests into different folders, then call tests option accordingly (i.e. use an environment variable or something)

Method 2: Use Tags in test names @onlyIE

You can give tests which should only run with Internet Explorer with the @onlyIE tag (i.e. add it to the scenario description).

# Only run tests for Internet Explorer
codeceptjs -c ./codecept.conf.ie.js --grep '@onlyIE'

# Only run tests for Chrome
codeceptjs -c ./codecept.conf.chrome.js --grep '@onlyChrome'

# Run tests on all tests which aren't browser specific i.e. don't have the `@only`
codeceptjs -c ./codecept.conf.default.js --grep '@only' --invert

In jenkins you can either run multiple commands and each codeceptjs has a different output folder, or you can also setup a multie configuration project but I found that to be a bit harder to track and maintain.

ninpop25 commented 6 years ago

@reubenmiller : Method 2 looks good but for this how we will setup codecept.json file?

I mean how to set config file to include both browsers[Chrome and IE] in below statement and just to call them at the time of Run of test by tag name.

"output": "./output", "helpers": { "WebDriverIO": { "url": "https://formsviewertest.azurewebsites.net", "browser": "chrome", "windowSize": "maximize", "restart": false, "keepBrowserState": true, "keepCookies": true },

I would also like you to tell that I run my tests by using this command (npm run codeceptjs)

Below is my Package.json file setup

{ "name": "fvautomation", "version": "1.0.0", "description": "", "main": "fv_anonymousTemplate_test.js", "dependencies": { "codeceptjs": "^1.0.3", "codeceptjs-webdriverio": "^1.1.0", "mocha": "^3.5.3", "mocha-junit-reporter": "^1.13.0", "mocha-multi": "^0.11.0", "mochawesome": "^2.2.0", "selenium-standalone": "^6.9.0", "sinon": "^2.3.5" }, "devDependencies": { "mocha": "^3.4.1" }, "scripts": { "test": "mocha", "codeceptjs": "codeceptjs run -R mocha-multi", "start": "node --max-old-space-size=2048 ./config/start_server.js" }, "author": "", "license": "ISC" }

reubenmiller commented 6 years ago

Refer to the docs. It should have all of the information you need to do this.

ninpop25 commented 6 years ago

@reubenmiller : Thanks, I have gone through this earlier also but whenever I configure Multiple suits in codeceptjs.json file I get below error while try to Run tests with this command (codeceptjs run-multiple Ie:internet explorer)

Multiple suites not configured, add "multiple": { /../ } section to config

Below is my config file

{ "output": "./output", "helpers": { "WebDriverIO": { "url": "https://formsviewertest.azurewebsites.net", "multiple": { "Ie": { "grep": "@Ie", "browser": [ "chrome", "internet explorer" ], "restart": true, "keepBrowserState": true, "keepCookies": true } } }, "TabSwitch": { "require": "./tabswitch_helper.js" } }, "include": { "I": "./steps_file.js", "loginPagePage": "./pages/loginPage.js", "manageCredentialPage": "./pages/manageCredential.js", "templatePage": "./pages/template.js", "uiHelper": "./pages/uiHelper.js", "fvAssert": "./pages/fvAssert.js", "listPage": "./pages/listPage.js", "fvData": "./InputData/login.js" }, "mocha": { "reporterOptions": { "codeceptjs-cli-reporter": { "stdout": "-", "options": { "verbose": false, "steps": true } }, "mocha-junit-reporter": { "stdout": "./output/console.log", "options": { "mochaFile": "./output/result.xml" } } } }, "bootstrap": "./config/start_server.js", "teardown": "./config/stopserver.js", "hooks": [], "tests": "**/fv*_test.js", "timeout": 10000, "name": "FV" }

reubenmiller commented 6 years ago

You've misplaced the multiple property. It should be a root property and not inside the WebDriverIO property. This is exactly what the error message is telling you.

{
    "output": "./output",
    "multiple": {
        "Ie": {
            "grep": "@ie",
            "browser": [
                "chrome",
                "internet explorer"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        }
    },
    "helpers": {
        "WebDriverIO": {
            "url": "https://formsviewertest.azurewebsites.net",
        },
        "TabSwitch": {
            "require": "./tabswitch_helper.js"
        }
    },
    "include": {
        "I": "./steps_file.js",
        "loginPagePage": "./pages/loginPage.js",
        "manageCredentialPage": "./pages/manageCredential.js",
        "templatePage": "./pages/template.js",
        "uiHelper": "./pages/uiHelper.js",
        "fvAssert": "./pages/fvAssert.js",
        "listPage": "./pages/listPage.js",
        "fvData": "./InputData/login.js"
    },
    "mocha": {
        "reporterOptions": {
            "codeceptjs-cli-reporter": {
                "stdout": "-",
                "options": {
                    "verbose": false,
                    "steps": true
                }
            },
            "mocha-junit-reporter": {
                "stdout": "./output/console.log",
                "options": {
                    "mochaFile": "./output/result.xml"
                }
            }
        }
    },
    "bootstrap": "./config/start_server.js",
    "teardown": "./config/stop_server.js",
    "hooks": [],
    "tests": "**/fv_*_test.js",
    "timeout": 10000,
    "name": "FV"
}
reubenmiller commented 6 years ago

But looking again at your config, I think you're missing the point. Here is a better config where two different profiles are configured.

{
    "output": "./output",
    "multiple": {
        "Ie": {
            "grep": "@ie",
            "browser": [
                "internet explorer"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        },
        "Chrome": {
            "grep": "@chrome",
            "browser": [
                "chrome"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        }
    },
    "helpers": {
        "WebDriverIO": {
            "url": "https://formsviewertest.azurewebsites.net",
        },
        "TabSwitch": {
            "require": "./tabswitch_helper.js"
        }
    },
    "include": {
        "I": "./steps_file.js",
        "loginPagePage": "./pages/loginPage.js",
        "manageCredentialPage": "./pages/manageCredential.js",
        "templatePage": "./pages/template.js",
        "uiHelper": "./pages/uiHelper.js",
        "fvAssert": "./pages/fvAssert.js",
        "listPage": "./pages/listPage.js",
        "fvData": "./InputData/login.js"
    },
    "mocha": {
        "reporterOptions": {
            "codeceptjs-cli-reporter": {
                "stdout": "-",
                "options": {
                    "verbose": false,
                    "steps": true
                }
            },
            "mocha-junit-reporter": {
                "stdout": "./output/console.log",
                "options": {
                    "mochaFile": "./output/result.xml"
                }
            }
        }
    },
    "bootstrap": "./config/start_server.js",
    "teardown": "./config/stop_server.js",
    "hooks": [],
    "tests": "**/fv_*_test.js",
    "timeout": 10000,
    "name": "FV"
}

Then you can run all profiles using

codeceptjs run-multiple --all

Now if you only want to run your tests relevant to internet explorer, then use

codeceptjs run-multiple Ie

Or chrome only tests run,

codeceptjs run-multiple Chrome
ninpop25 commented 6 years ago

@reubenmiller : Thanks for writing the complete config file.I used the same and now when Run my test by using this command (codeceptjs run-multiple Ie) then I am getting below error

D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>c odeceptjs run-multiple Ie

C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\codeceptjs\lib\command\ru n-multiple.js:74 if (suiteConf.browsers.length === 0) throw new Error(Browser ${browser} not found in multiple suite "${suiteName}" config); ^

TypeError: Cannot read property 'length' of undefined at C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\codeceptjs\lib\com mand\run-multiple.js:74:27 at Array.forEach (native) at Command.module.exports (C:\Users\Daffolap-254\AppData\Roaming\npm\node_mo dules\codeceptjs\lib\command\run-multiple.js:50:10) at Command.listener (C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\ codeceptjs\node_modules\commander\index.js:300:8) at emitTwo (events.js:125:13) at Command.emit (events.js:213:7) at Command.parseArgs (C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules \codeceptjs\node_modules\commander\index.js:635:12) at Command.parse (C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\cod eceptjs\node_modules\commander\index.js:457:21) at Object. (C:\Users\Daffolap-254\AppData\Roaming\npm\node_module s\codeceptjs\bin\codecept.js:107:9) at Module._compile (module.js:569:30)

reubenmiller commented 6 years ago

Looking at the docs it should be browsers not browser.

i.e.

"browsers": [
   "internet explorer"
],

I just copied what you posted earlier, I didn't run it myself.

ninpop25 commented 6 years ago

@reubenmiller : Thanks for writing the complete config file.I used the same and now when Run my test by using this command (codeceptjs run-multiple Ie) then I am getting below error

D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>c odeceptjs run-multiple Ie

C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\codeceptjs\lib\command\ru n-multiple.js:74 if (suiteConf.browsers.length === 0) throw new Error(Browser ${browser} not found in multiple suite "${suiteName}" config); ^

TypeError: Cannot read property 'length' of undefined at C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\codeceptjs\lib\com mand\run-multiple.js:74:27 at Array.forEach (native) at Command.module.exports (C:\Users\Daffolap-254\AppData\Roaming\npm\node_mo dules\codeceptjs\lib\command\run-multiple.js:50:10) at Command.listener (C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\ codeceptjs\node_modules\commander\index.js:300:8) at emitTwo (events.js:125:13) at Command.emit (events.js:213:7) at Command.parseArgs (C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules \codeceptjs\node_modules\commander\index.js:635:12) at Command.parse (C:\Users\Daffolap-254\AppData\Roaming\npm\node_modules\cod eceptjs\node_modules\commander\index.js:457:21) at Object. (C:\Users\Daffolap-254\AppData\Roaming\npm\node_module s\codeceptjs\bin\codecept.js:107:9) at Module._compile (module.js:569:30)

ninpop25 commented 6 years ago

@reubenmiller : Dont know but its not working for me even after correct config now. after changing browser to browsers I again run the test and then got the error that .helper/wedriverio needs both Url and Browser to be defined,so I defined the browser in it to be chrome,then I downloaded the latest IEdriverServer.exe and placed to my project folder.

Then when I run the test I got another error that all the Security zones in IE must be set to same level ,so I just did that and again Run the test.

Now I am getting this below error which I think is related to version of IEDriverServer.exe so I downgraded the version but again I am getting the below error.

 _Invalid timeout type specified: ms

Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:54:15 .039Z' System info: host: 'DAFFOLAP-254-PC', ip: '192.168.43.27', os.name: 'Windows 7', os.arch: 'x86', os.version: '6.1', java.version: '1.8.0131' Driver info: driver.version: unknown


Now My Config looks like below

{ "output": "./output", "multiple": { "Ie": { "grep": "@ie", "browsers": [ "internet explorer" ], "restart": true, "keepBrowserState": true, "keepCookies": true }, "Chrome": { "grep": "@chrome", "browsers": [ "chrome" ], "restart": true, "keepBrowserState": true, "keepCookies": true } }, "helpers": { "WebDriverIO": { "url": "https://formsviewertest.azurewebsites.net", "browser": "chrome" (I added this one too as without this I was getting the error that I defined above in starting comments) }, "TabSwitch": { "require": "./tabswitch_helper.js" } }, "include": { "I": "./steps_file.js", "loginPagePage": "./pages/loginPage.js", "manageCredentialPage": "./pages/manageCredential.js", "templatePage": "./pages/template.js", "uiHelper": "./pages/uiHelper.js", "fvAssert": "./pages/fvAssert.js", "listPage": "./pages/listPage.js", "fvData": "./InputData/login.js" }, "mocha": { "reporterOptions": { "codeceptjs-cli-reporter": { "stdout": "-", "options": { "verbose": false, "steps": true } }, "mocha-junit-reporter": { "stdout": "./output/console.log", "options": { "mochaFile": "./output/result.xml" } } } }, "bootstrap": "./config/start_server.js", "teardown": "./config/stopserver.js", "hooks": [], "tests": "**/fv*_test.js", "timeout": 10000, "name": "FV" }

Please look into this

reubenmiller commented 6 years ago

This is working for me on Windows 7.

package.json

{
  "name": "doler-js",
  "version": "0.0.1",
  "scripts": {
    "test:multi_all": "codeceptjs run-multiple --all -c ./codecept-config/codecept.multiple.js",
    "test:multi_chrome": "codeceptjs run-multiple chrome:chrome -c ./codecept-config/codecept.multiple.js",
    "test:multi_ie": "codeceptjs run-multiple \"ie:internet explorer\" -c ./codecept-config/codecept.multiple.js",
  }
}

codecept.multiple.js

exports.config = {
    output: './output',

    multiple: {
        ie: {
            "grep": "@ie",
            "browsers": [
                "internet explorer"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        },
        chrome: {
            "grep": "@chrome",
            "browsers": [
                "chrome"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        },
    },
    helpers: {
        WebDriverIO: {
            url: 'https://formsviewertest.azurewebsites.net',
            browser: 'chrome',
        },
        FileSystem: {},
    },
    include: {},
    mocha: {},
    bootstrap: false,
    teardown: null,
    hooks: [],
    tests: '../tests-multiple/*_test.js',
    timeout: 10000,
    name: 'doler-js',
};

Commands (from the scripts section of the package.json)

# Internet explorer only tests
npm run test:multi_ie
# Chrome only tests
npm run test:multi_chrome
# All Tests
npm run test:multi_all

Internet Explorer config

Disable-InternetExplorerZones.ps1

Param(
    [string[]] $ComputerName = "127.0.0.1",

    [int] $IEMode = 3
)

foreach ($Computer in $ComputerName) {
    #Test connectivity first
    $Connected = Test-Connection -computer $Computer -quiet -count 2

    if ($Connected)
    {
        Write-Host "Checking Internet Explorer Zone settings on [$Computer]"
        #
        # Check if the remote registry service is running (required for the script to work)
        #
        $RemoteRegistryService = Get-Service -Name RemoteRegistry -ComputerName $Computer -ErrorAction SilentlyContinue

        if ($RemoteRegistryService) {
            if ($RemoteRegistryService.Status -ne "running") {
                Write-Host "Starting Windows RemoteRegistry Service because it was not already running"
                $RemoteRegistryService.Start()
                $RemoteRegistryService.WaitForStatus("running", "00:00:05")
            }
        }

        # Collecting all users that are currently having Registry hives loaded
        $reg = Get-WmiObject -List -Namespace root\default -ComputerName $Computer | Where-Object {$_.Name -eq "StdRegProv"}
        $HKEY_USERS = 2147483651
        $Users = $reg.EnumKey($HKEY_USERS, "") | Select-Object -ExpandProperty sNames

        foreach ($User in $Users)
        {
            #$User.Length

            # We are only intersted in AD users. Their SID lengh is 44 or 45 symbols
            # if ($User -ne $null)
            if (($User.Length -eq 45) -or ($User.Length -eq 44))
            {
                Write-Host "User: $User"

                # [Microsoft.Win32.RegistryHive]::Users
                $reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($HKEY_USERS, $Computer)
                $pathV = $User + "\Volatile Environment"
                $regkey = $reg.OpenSubkey($pathV)

                # If a user does not have the registry key, then we are not going to check him
                if ($regkey -ne $null)
                {

                    # Getting User name
                    $Username = $regkey.GetValue("USERNAME")
                    "User:" + $Username

                    # Getting Zones
                    $pathZ = $User + "\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\"
                    $regkey11 = $reg.OpenSubkey($pathZ)
                    $Zones = $regkey11.GetSubKeyNames()

                    # For each zone extract "Protected Mode" state
                    if ($regkey11 -ne $null)
                    {
                        $Modes = @()
                        foreach ($Zone in $Zones)
                        {
                            $DPath = $pathZ + '\' + $Zone;
                            $regN = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey($HKEY_USERS, $Computer).OpenSubkey($DPath, $true)  # Open key in writable mode, hence the $true
                            [int] $CurrentMode = $regN.GetValue("2500")
                            'Zone ' + $Zone + ': ' + $regN.GetValue("2500")
                            $Modes += $regN.GetValue("2500")

                            if (($Zone -gt 0) -and ($CurrentMode -ne $IEMode)) {
                                Write-Host "Setting Zone [$Zone] to Mode [$IEMode], current Mode is [$CurrentMode]"
                                $regN.SetValue("2500", $IEMode, [Microsoft.Win32.RegistryValueKind]::DWORD)
                            }

                        }
                    }
                }

            }

        }
    }

}
ninpop25 commented 6 years ago

@reubenmiller : I really appreciate your help on this,but I think this is not going to work for me. I did everything as told by you but then also m getting error.

Below is my Package.json

_{ "name": "fvautomation", "version": "1.0.0", "description": "", "main": "fv_anonymousTemplate_test.js", "dependencies": { "codeceptjs": "^1.0.3", "codeceptjs-webdriverio": "^1.1.0", "log4js": "^2.5.2", "mocha": "^3.5.3", "mocha-junit-reporter": "^1.13.0", "mocha-multi": "^0.11.0", "mochawesome": "^2.2.0", "selenium-standalone": "^6.9.0", "sinon": "^2.3.5" }, "devDependencies": { "mocha": "^3.4.1" }, "scripts": { "test": "mocha", "codeceptjs": "codeceptjs run -R mocha-multi", "start": "node --max-old-space-size=2048 ./config/start_server.js", "test:multi_all": "codeceptjs run-multiple --all -c ./codecept-config/codecept.multiple.js", "test:multi_chrome": "codeceptjs run-multiple chrome:chrome -c ./codecept-config/codecept.multiple.js", "test:multi_ie": "codeceptjs run-multiple \"ie:internet explorer\" -c ./codecept-config/codecept.multiple.js" }, "author": "", "license": "ISC"

}_

Below is my Config File

_{ "output": "./output", "multiple": { "ie": { "grep": "@ie", "browsers": [ "internet explorer" ], "restart": true, "keepBrowserState": true, "keepCookies": true }, "Chrome": { "grep": "@chrome", "browsers": [ "chrome" ], "restart": true, "keepBrowserState": true, "keepCookies": true } }, "helpers": { "WebDriverIO": { "url": "https://formsviewertest.azurewebsites.net", "browser": "chrome" }, "FileSystem": {}, "TabSwitch": { "require": "./tabswitch_helper.js" } }, "include": { "I": "./stepsfile.js", "loginPagePage": "./pages/loginPage.js", "manageCredentialPage": "./pages/manageCredential.js", "templatePage": "./pages/template.js", "uiHelper": "./pages/uiHelper.js", "fvAssert": "./pages/fvAssert.js", "listPage": "./pages/listPage.js", "fvData": "./InputData/login.js" }, "mocha": { "reporterOptions": { "codeceptjs-cli-reporter": { "stdout": "-", "options": { "verbose": false, "steps": true } }, "mocha-junit-reporter": { "stdout": "./output/console.log", "options": { "mochaFile": "./output/result.xml" } } } }, "bootstrap": false, "teardown": null, "hooks": [], "tests": "../tests-multiple/fv*test.js", "timeout": 10000, "name": "fvautomation" }

Below is the error I am getting

_D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>n pm run test:multi_ie

fvautomation@1.0.0 test:multi_ie D:\FV_Automation_Debug\Projects\Qdabra\Source Code\AutomationTestScripts\Source codeceptjs run-multiple "ie:internet explorer" -c ./codecept-config/codecept.m ultiple.js

module.js:487 throw err; ^

Error: Cannot find module 'codeceptjs/bin/codecept' at Function.Module._resolveFilename (module.js:485:15) at Function.Module._load (module.js:437:25) at Module.require (module.js:513:17) at require (internal/module.js:11:18) at Object. (D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\Aut omationTestScripts\Source\node_modules\codeceptjs-webdriverio\codecept.js:2:1) at Module._compile (module.js:569:30) at Object.Module._extensions..js (module.js:580:10) at Module.load (module.js:503:32) at tryModuleLoad (module.js:466:12) at Function.Module._load (module.js:458:3) npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! fvautomation@1.0.0 test:multi_ie: codeceptjs run-multiple "ie:internet explorer" -c ./codecept-config/codecept.multiple.js npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the fvautomation@1.0.0 test:multi_ie script. npm ERR! This is probably not a problem with npm. There is likely additional log ging output above.

npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\Daffolap-254\AppData\Roaming\npm-cache_logs\2018-01-27T18 _37_14_637Z-debug.log

D:\FV_AutomationDebug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>

don't know what wrong I am doing here.

I am also using window 7

reubenmiller commented 6 years ago

Though you can't just copy paste the stuff in there because it will have the paths relevant to my project in there, and not yours.

In my project my codceptpjs config is a .js file under ./codecept-config/codcept.multiple.js. So I am assuming that this path does not exist for you and that is why the error is occurring.

You can't put a slimmed down version of your project on github?

reubenmiller commented 6 years ago

Did you also see my note about the codeceptjs.bat earlier? Did you try creating this small batch wrapper and placing it in the same directory as the package.json?

ninpop25 commented 6 years ago

@reubenmiller : sorry ,yes your are right that I should not share the config nd package of my project on GitHub,but it's a debugging copy nd actual is somewhat different but then also I should not share ,will keep this in mind.. thanks.

Yes I saw your message regarding .bat file, I am not so good in that,but I will try .

I am little bit confused by your below comments.

In my project my codceptpjs config is a .js file under ./codecept-config/codcept.multiple.js. So I am assuming that this path does not exist for you and that is why the error is occurring.

My codecept config is Jason file,so how should I proceed?

reubenmiller commented 6 years ago

ok, so I've edited your files so it should fit with your project structure (I hope)...

package.json

{
    "name": "fvautomation",
    "version": "1.0.0",
    "description": "",
    "main": "fv_anonymousTemplate_test.js",
    "dependencies": {
        "codeceptjs": "^1.0.3",
        "codeceptjs-webdriverio": "^1.1.0",
        "log4js": "^2.5.2",
        "mocha": "^3.5.3",
        "mocha-junit-reporter": "^1.13.0",
        "mocha-multi": "^0.11.0",
        "mochawesome": "^2.2.0",
        "selenium-standalone": "^6.9.0",
        "sinon": "^2.3.5"
    },
    "devDependencies": {
        "mocha": "^3.4.1"
    },
    "scripts": {
        "test": "mocha",
        "codeceptjs": "codeceptjs run -R mocha-multi",
        "start": "node --max-old-space-size=2048 ./config/start_server.js",
        "test:multi_all": "codeceptjs run-multiple --all",
        "test:multi_chrome": "codeceptjs run-multiple chrome:chrome",
        "test:multi_ie": "codeceptjs run-multiple \"ie:internet explorer\""
    },
    "author": "",
    "license": "ISC"

}

Codeceptjs config (your json file)

{
    "output": "./output",
    "multiple": {
        "ie": {
            "grep": "@ie",
            "browsers": [
                "internet explorer"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        },
        "chrome": {
            "grep": "@chrome",
            "browsers": [
                "chrome"
            ],
            "restart": true,
            "keepBrowserState": true,
            "keepCookies": true
        }
    },
    "helpers": {
        "WebDriverIO": {
            "url": "https://formsviewertest.azurewebsites.net",
            "browser": "chrome"
        },
        "TabSwitch": {
            "require": "./tabswitch_helper.js"
        }
    },
    "include": {
        "I": "./steps_file.js",
        "loginPagePage": "./pages/loginPage.js",
        "manageCredentialPage": "./pages/manageCredential.js",
        "templatePage": "./pages/template.js",
        "uiHelper": "./pages/uiHelper.js",
        "fvAssert": "./pages/fvAssert.js",
        "listPage": "./pages/listPage.js",
        "fvData": "./InputData/login.js"
    },
    "mocha": {
        "reporterOptions": {
            "codeceptjs-cli-reporter": {
                "stdout": "-",
                "options": {
                    "verbose": false,
                    "steps": true
                }
            },
            "mocha-junit-reporter": {
                "stdout": "./output/console.log",
                "options": {
                    "mochaFile": "./output/result.xml"
                }
            }
        }
    },
    "bootstrap": "./config/start_server.js",
    "teardown": "./config/stop_server.js",
    "hooks": [],
    "tests": "**/fv_*_test.js",
    "timeout": 10000,
    "name": "FV"
}
reubenmiller commented 6 years ago

I've created a demo project showing a working example.

I haven't added any custom helpers or page objects to the project, but at least it should show you how to do the multi configuration settings. Note it does not use the codeceptjs json file, it uses the javascript version codecept.conf.js instead.

ninpop25 commented 6 years ago

@reubenmiller : Thanks for sharing the example,I cloned it and followed the steps you mentioned and guess what it is working.

But when I am doing with my codecept.json and package file that you edited for me then I am again getting the same error as below

_D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>n pm run test:multi_ie

fvautomation@1.0.0 test:multi_ie D:\FV_Automation_Debug\Projects\Qdabra\Source Code\AutomationTestScripts\Source codeceptjs run-multiple "ie:internet explorer"

module.js:487 throw err; ^

Error: Cannot find module 'codeceptjs/bin/codecept' at Function.Module._resolveFilename (module.js:485:15) at Function.Module._load (module.js:437:25) at Module.require (module.js:513:17) at require (internal/module.js:11:18) at Object. (D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\Aut omationTestScripts\Source\node_modules\codeceptjs-webdriverio\codecept.js:2:1) at Module._compile (module.js:569:30) at Object.Module._extensions..js (module.js:580:10) at Module.load (module.js:503:32) at tryModuleLoad (module.js:466:12) at Function.Module._load (module.js:458:3) npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! fvautomation@1.0.0 test:multi_ie: codeceptjs run-multiple "ie:internet explorer" npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the fvautomation@1.0.0 test:multi_ie script. npm ERR! This is probably not a problem with npm. There is likely additional log ging output above.

npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\Daffolap-254\AppData\Roaming\npm-cache_logs\2018-01-28T07 _42_48_477Z-debug.log

D:\FV_AutomationDebug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>

don't know why its not working for me again and again getting same error

reubenmiller commented 6 years ago
ninpop25 commented 6 years ago

@reubenmiller : the demo project you gave me,it is working good with your files.

But when I am working with my project folder with my package file and codeceptjs json file that you gave me yesterday after editing as per my project folder then I am getting the error.

reubenmiller commented 6 years ago

Yeah so it must be something strange with your project setup...so that narrows it down at least.

Next steps

ninpop25 commented 6 years ago

@reubenmiller : ok done and now _./nodemodules/codeceptjs/bin/codecept.js its exits :) Now I am getting this error Cannot find module 'appdynamics' Should I install this by npm install appdynamics?

complete error is below.

D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>n pm run test:multi_ie

fvautomation@1.0.0 test:multi_ie D:\FV_Automation_Debug\Projects\Qdabra\Source Code\AutomationTestScripts\Source codeceptjs run-multiple "ie:internet explorer"

creating output directory: D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\Aut omationTestScripts\Source\output\iebrowser_internet_explorer1 [1.ie:internet explorer] [1.ie:internet explorer] Cannot find module 'appdynamics' [1.ie:internet explorer] [1.ie:internet explorer] Error: at Function.Module._resolveFilename (module.js:485:15) at Function.Module._load (module.js:437:25) at Module.require (module.js:513:17) at require (internal/module.js:11:18) at Object. (D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\Aut omationTestScripts\Source\config\start_server.js:1:63) at Module._compile (module.js:569:30) at Object.Module._extensions..js (module.js:580:10) at Module.load (module.js:503:32) at tryModuleLoad (module.js:466:12) at Function.Module._load (module.js:458:3) npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! fvautomation@1.0.0 test:multi_ie: codeceptjs run-multiple "ie:internet explorer" npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the fvautomation@1.0.0 test:multi_ie script. npm ERR! This is probably not a problem with npm. There is likely additional log ging output above.

npm ERR! A complete log of this run can be found in: npm ERR! C:\Users\Daffolap-254\AppData\Roaming\npm-cache_logs\2018-01-28T08 _13_49_604Z-debug.log

reubenmiller commented 6 years ago

Yeah install it with npm, it has nothing to do with codeceptjs, but you've probably used it somewhere in your project.

ninpop25 commented 6 years ago

@reubenmiller : Finally its working for me, thank you so much my dear friend :) ,this was great great help as you have been continuously responding my queries on this particular issue specially

one update I will would like to give you that with latest version of IEDriverServer 3.8.0 its throwing some error related to "Invalid timeout type specified", So I tried just downloading the downgraded version 2.53.1 and it worked for me.

Note : Strange is that my test is getting launched in IE but its not able to find the element that I am locating via xpath ..its working fine for me in Chrome but don't know why its failing with IE

Query : Also I want to know that with this package.json and codecept json file in which we have specified multiple function and then using command to run test that contains @ie in it to run in IE

Can I still use the command npm run codeceptjs to run rest of my tests in Chrome?

reubenmiller commented 6 years ago

Cool 👍

As for your question: From your previous package.json, yeah it looks like npm run codeceptjs should run all of the tests in chrome.

Just in case if your package.json has changed since your last post, the following command line should work (note: I favour the verbose argument names, --reporter instead of -R, because it is easier to understand what the argument means.)

package.json Extract

{
  "scripts": {
    "codeceptjs": "codeceptjs run --reporter mocha-multi"
  }
}

The script runs codeceptjs in single configuration mode (not multiple configuration mode). So the browser used in this mode is controlled by the config in the Helpers section.

And here you have chrome defined as the browser to use for WebDriverIO

"helpers": {
        "WebDriverIO": {
            "url": "https://formsviewertest.azurewebsites.net",
            "browser": "chrome"
        }
    }
ninpop25 commented 6 years ago

@reubenmiller : I will use reporter instead of -R

This was the great help :)

If you had a time to see this : https://github.com/Codeception/CodeceptJS/issues/892 then please help me in this too :)

ninpop25 commented 6 years ago

@reubenmiller : Sorry I am reopening the issue as I have one last query related to this implementation.

How can I avoid mom run codeceptjs to not Run the tests that has @ie Tag in it?

As I have total 220 tests and out of which there are arround 4 cases that I want to Run on IE.

I am thinking of using first npm run codeceptjs in Jenkins as executable batch command and then I will use second executable batch command after this and that will be npm run multi:ie

But with first batch command I want it to run tests that does not have any Tag,but I think it Will Run other tests also that has Tag ie in it.

So how to avoid npm run codeceptjs to not run tests with Tag ie

ninpop25 commented 6 years ago

@reubenmiller : I think I miss type it

It's npm run codeceptjs

I typed npm as mom

reubenmiller commented 6 years ago

It looks like the docs don't explicitly mention the --invert option. I found this by looking through the code.

If you use the --grep option you can actually invert the match by also specifying --invert.

So if you want to run all tests in single configuration mode, but exclude all of the tests marked with the @ie tag, then the command looks like this.

"scripts": {
  "test:no_ie": "codeceptjs run --grep @ie --invert --reporter mocha-multi"
}

Running the script using:

npm run test:no_ie
ninpop25 commented 6 years ago

@reubenmiller : Cool this worked for me now another issue

When I Run selenium server manually by using this command [java -jar -Dwebdriver.chrome.driver=chromedriver.exe selenium-server-standalone-3.8.1.jar ] then I am getting below error

D:\FV_Automation_Debug\Projects\Qdabra\SourceCode\AutomationTestScripts\Source>n pm run test:multi_ie

fvautomation@1.0.0 test:multi_ie D:\FV_Automation_Debug\Projects\Qdabra\Source Code\AutomationTestScripts\Source codeceptjs run-multiple "ie:internet explorer"

null [1.ie:internet explorer] CodeceptJS v1.1.2 [1.ie:internet explorer] Using test root "D:\FV_Automation_Debug\Projects\Qdabra \SourceCode\AutomationTestScripts\Source"

[1.ie:internet explorer] Fv anonymousShpntSbmt -- [1.ie:internet explorer] x "before all" hook: codeceptjs.beforeSuite in 3053ms [1.ie:internet explorer] TypeError: Cannot read property '0' of null

[1.ie:internet explorer] -- FAILURES:

1) Fv anonymousShpntSbmt "before all" hook: codeceptjs.beforeSuite: Cannot read property '0' of null


I then just stopped the selenium server for chrome and then Run the same for IE by using this command [java -jar -Dwebdriver.ie.driver=IEDriverServer.exe selenium-server-standalone-2.53.1.jar] and the Run my IE only test and found its working then.

Query : So that means I have to make sure that selenium server is Up for both Chrome and IE ,but how this can be done as we can't run 2 server at same time.

reubenmiller commented 6 years ago

We should really try and keep these issues separate, as we are now straying from the original problem.

But when starting selenium server it is best to configures all of the webdrivers that you require (i.e. chromedriver, iedriver, geckodriver (firefox)).

I believe you can specify the paths to the appropriate drivers by using -D arguments, for Example:

-Dwebdriver.gecko.driver="yourFolder/0.16.0-x64-geckodriver.exe" -Dwebdriver.chrome.driver="yourFolder/2.29-x64-chromedriver.exe" -Dwebdriver.ie.driver="yourFolder/3.4.0-x64-IEDriverServer.exe"

Though please note I haven't tested this, so please don't copy/paste it. But it gives you an idea what is required. Though you get rid of all of this hassel if you you use the npm project selenium-standalone.

For future info about using selenium server please hit up the selenium server repo.

ninpop25 commented 6 years ago

Thanks @reubenmiller : earlier also I was using this https://github.com/vvo/selenium-standalone method to start the selenium server automatically but after the update of Chrome it was not working so I switched to manually running the selenium server.

Yesterday I again followed all 3 steps i.e. npm install selenium-standalone@latest -g selenium-standalone install selenium-standalone start

and I now my selenium server starts automatically when I run my tests on my local machine.

Issue is that when I am doing same on my client machine (Windows 10) Via Jenkins by using this command (npm run codeceptjs) then I saw that although the tests starts Run on jenkins and I can see them in jenkins console but when I check on my client server then I found that there was no Chrome browser that has been launched so I guess that tests are running Headless

Dont know why its not launching chrome browser on my client machine.

Please click on this link to where I have placed all required configs which can help you to debug the issue

reubenmiller commented 6 years ago

I don't think that has anything to do with codeceptjs, and is more of a jenkins issue.

However I use a similar setup, and I think the problem is caused by lack of permissions by the user which is used to launch the jenkins tests (i.e. the SYSTEM user by default).

So what I ended up doing was to create a Windows service (using NSSM). I personally use the Interactive services option so the browser does not appear on the desktop session but a dialog box is shown which allows the user to view the session where the tests are being executed".

Once this service has been created, I just start the Windows service before executing my codeceptjs tests.

Here is a powershell script that I created (as is), so you can get an idea what is required.

The script assumes that you have the following in the same folder (or sub folder) as the .ps1 file:

# Note you will have to update the web drivers manually, and place them in the relevant InstallPath before creating the service.
$Args = @(
  "-Dwebdriver.gecko.driver=""$InstallPath/0.16.0-x64-geckodriver""",
  "-Dwebdriver.chrome.driver=""$InstallPath/2.29-x64-chromedriver""",
  "-Dwebdriver.ie.driver=""$InstallPath/3.4.0-x64-IEDriverServer.exe""",
  "-jar ""$JarFullPath"""
)

You can install the service by running the following in a Powershell console (powershell.exe should be launched like powershell.exe -ExecutionPolicy bypass)

. .\SeleniumService.ps1
Install-SeleniumService -InstallPath "C:\MyInstallPath"
ninpop25 commented 6 years ago

@reubenmiller : I just looked in jenkins services,Please check below image and let me know that clicking on highlighted check box will launch the browser?

Although I will follow your steps too

image

reubenmiller commented 6 years ago

I don't think so, it probably is more related to the selenium server process (hence why I created a service for it so I could control it separately).

Though I checked the jenkins agent (I use a jenkins slave node to execute my tests), and it doesn't have the interactive services checked, it is only my selenium server.

But will all things I only found a solution that worked for my by trying different things out.

reubenmiller commented 6 years ago

Closing this issue as it is more related to the selenium setup and integrating it with Jenkins.

This type of questions might be more suited to the slack channel.

ajmalkhan92 commented 6 years ago

It still exists in codecept with Selenium standalone 3.4.0 with IE driver

Logs:- [3.basic:internet explorer] Error: Invalid timeout type specified: ms Build info: version: '3.7.1', revision: '8a0099a', time: '2017-11-06T21:07:36.16 1Z' System info: host: 'W2694290BYYDX', ip: '172.20.46.245', os.name: 'Windows 7', o s.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_144' Driver info: driver.version: unknown

0 passing (5s) 1 failing

1) My First Test "before each" hook: codeceptjs.before for "login": Invalid timeout type specified: ms Build info: version: '3.7.1', revision: '8a0099a', time: '2017-11-06T21:07:36.16 1Z' System info: host: 'W2694290BYYDX', ip: '172.20.46.245', os.name: 'Windows 7', o s.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_144' Driver info: driver.version: unknown

reubenmiller commented 6 years ago

From what I can tell it is probably an issue with the ie driver, so we can't do much about it unfortunately. Just try different iedriver versions and maybe the 32bit or 64bit etc.