yahoo / protractor-retry

Use protractor features to automatically re-run failed tests with a specific configurable number of attempts.
MIT License
28 stars 32 forks source link

protractor-retry don't works on "ng cli" projects #6

Closed karlos1337 closed 7 years ago

karlos1337 commented 7 years ago

Hi, First of all, I would like to tell you that this package would solve a huge problem that I have been carrying on for years, so thank you very much for your work.

At the moment I'm working with an angular project based on its official CLI "ng cli", this replaces the npm task manager as a CLI and the entire workflow passes through it, including e2e tests execution, after installing and configuring protractor-retry effectively the tests that fail are repeated but the process never ends nor does returns the satisfactory output that I need for my CI, the following error appears:

The option '--retry' is not registered with the e2e command. Run `ng e2e --help` for a list of supported options.
The option '--disableChecks' is not registered with the e2e command. Run `ng e2e --help` for a list of supported options.

It is not clear to me that this can be solved from your module without editing the "ng cli" code, but if possible it would solve a big headache for me.

dreuxl commented 7 years ago

Hi Thanks for your feedback. We wish your integration went smooth though :)

So couple of questions regarding your issue to try to isolate it:

I have the 2nd one (--disableChecks) lately on my own runs but which is coming as a warning only, something like

[22:55:41] W/runner - Ignoring unknown extra flags: test, retry, testSuite. This will be an error in future versions, please use --disableChecks flag to disable the  Protractor CLI flag checks. 
karlos1337 commented 7 years ago

The output remains open in console until Ctrl+C so I can't use it with Jenkins or any CI.

Thanks.

amrot17 commented 7 years ago

Hi ,

This is not something we can control from our side , since sending flags to ng cli e2e command that are not recognized by the ng cli will throws an error.

checking the links below you'll find that ng cli was not supporting some important and heavily used flags by protractor like ( baseUrl , suites ) and they got requested and implemented.

https://github.com/angular/angular-cli/pull/3551/files https://github.com/angular/angular-cli/issues/5705

So i think that ng cli should implement something like (disableChecks) to allow cli commands that not recognized to pass same as protractor behavior.

(disableChecks) is a flag used for any protractor command that contains flags not recognized by protractor CLI, and this change introduced on protractor@5 , and we are leveraging this feature to implement our retry mechanism.

Copying the text from protractor@5 change log https://github.com/angular/protractor/blob/master/CHANGELOG.md#500

Sending flags that are not recognized by the CLI throws an error. Since flags are a subset of all configuration options, these errors can be silenced with --disableChecks.

Thanks

karlos1337 commented 7 years ago

All right, I suppose so, I have opened an issue in the ng-cli repository asking them to implement that flag and referencing this issue.

Thank you very much.