NickTomlin / protractor-flake

Rerun potentially flakey protractor tests before failing.
MIT License
80 stars 51 forks source link

retryconfig doesn't work #93

Open Anyman552 opened 6 years ago

Anyman552 commented 6 years ago

Hi there!

everything works fine until i want to use the retryconfig-feature. The first test run works well. But if there is a failure i get an error at the next test attempt:

Using standard to parse output
Re-running tests: test attempt 2
Using provided protractorRetryConfig: C:\seu\workspace\src\test\javascript\retryconfig.js
Re-running the following test files:
C:\seu\workspace\src\test\javascript\e2e\test\startbarkeitAuskunftProzesse.js
Usage: protractor [configFile] [options]
configFile defaults to protractor.conf.js
The [options] object will override values from the config file.
See the reference config for a full list of options.

Options:
  --help                                 Print Protractor help menu
  --version                              Print Protractor version
  --browser, --capabilities.browserName  Browsername, e.g. chrome or firefox
  --seleniumAddress                      A running selenium address to use
  --seleniumSessionId                    Attaching an existing session id
  --seleniumServerJar                    Location of the standalone selenium jar file
  --seleniumPort                         Optional port for the selenium standalone server
  --baseUrl                              URL to prepend to all relative paths
  --rootElement                          Element housing ng-app, if not html or body
  --specs                                Comma-separated list of files to test
  --exclude                              Comma-separated list of files to exclude
  --verbose                              Print full spec names
  --stackTrace                           Print stack trace on error
  --params                               Param object to be passed to the tests
  --framework                            Test framework to use: jasmine, mocha, or custom
  --resultJsonOutputFile                 Path to save JSON test result
  --troubleshoot                         Turn on troubleshooting output
  --elementExplorer                      Interactively test Protractor commands
  --debuggerServerPort                   Start a debugger server at specified port instead of repl
  --disableChecks                        Disable cli checks
  --logLevel                             Define Protractor log level [ERROR, WARN, INFO, DEBUG]

Error: Error: more than one config file specified

Protractor seems to throw the error because of more than one config files. So i looked at protractorArgs in index.js:

    if (parsedOptions.protractorRetryConfig && retry) {
      protractorArgs.push(parsedOptions.protractorRetryConfig);
    }

it adds the retryconfig but do not remove the origin one. If i remove the origin config from the array everything works fine:

    if (parsedOptions.protractorRetryConfig && retry) {
      protractorArgs.push(parsedOptions.protractorRetryConfig);
      protractorArgs.splice(1, 1);
    }

protractorFlake({ protractorPath: 'C:\Users\Anyman552\AppData\Roaming\npm\node_modules\protractor\bin\protractor', maxAttempts: 3, parser: 'standard', // expects node to be in path // set this to wherever the node bin is located nodeBin: 'node', // set color to one of the colors available at 'chalk' - https://github.com/chalk/ansi-styles#colors color: 'magenta', protractorArgs: ["protractor.conf.local.js"], // specify a different protractor config to apply after the first execution attempt // either specify a config file, or cli args (ex. --capabilities.browser=chrome) protractorRetryConfig: 'retryconfig.js' }, function (status, output) { process.exit(status); });

- protractor-config:
```js
exports.config = {
    framework: 'jasmine',
    specs: ["e2e/test/*.js"],
    directConnect: true,
    capabilities: {
        browserName: 'firefox',
            'moz:firefoxOptions': {
            args: [ "--headless" ]
         }
    },
    params: {
        e2e: {
            'config': {
                'helper': '../../e2e/mods/helper',
            }
        }
    }
};

the retryconfig is the same as the origin one with some other params.

Thanks, Anyman552

NickTomlin commented 6 years ago

Ah this definitely seems like a bug. Thank you for the thorough issue!

We could go with the approach you've suggested and just document that users must pass the config as the first option in protractorArgs. I'd want to make sure that we aren't accidentally removing too much though.

Would you be up for submitting a PR with some tests? I'm currently pretty swamped with my day job but i'd be happy to collaborate on a fix.

Anyman552 commented 6 years ago

I'm not sure if passing the config as first option is the solutution.

I'll take a look on it and submit a PR.

Anyman552 commented 6 years ago

I've submitted a PR: #94