mlison / protractor-jasmine2-screenshot-reporter

Protractor screenshot reporter for Jasmine2
https://www.npmjs.com/package/protractor-jasmine2-screenshot-reporter
MIT License
81 stars 79 forks source link

Ignore Skipped Specs Doesn't Respect Use of "fit" / "fdescribe" #68

Open Stephen-Williams opened 8 years ago

Stephen-Williams commented 8 years ago

Issue: If I set "ignoreSkippedSpecs" to "true" and then, in my spec files, use "fit" or "fdescribe" to only run a single test/block of tests, the screenshot reporter still creates screenshots for all of the tests in my suite, even though only a fraction of those are being executed.

For example, I have a suite with 82 tests. I only run 3 of those, by marking the desired tests with "fit". Protractor only executes the 3 marked tests, skipping everything else, but the screenshot reporter puts 82 screenshots in the destination directory, and creates a report that references all 82 tests as though they had all been run. Most of these screenshots are not at all appropriate to the tests they're mapped to, because, of course, those tests weren't actually executed.

I'm guessing this is because the "ignore" logic is looking specifically for "xit" and "xdescribe" to decide if a test is being skipped. When I use "fit" to run a single test, I don't make any changes to the other tests in the suite - Protractor skips them at runtime.

Here's my config for the reporter:

var HtmlReporter = new HtmlScreenshotReporter({
  dest: "/tmp",         
  filename: "my-report.html",                  
  ignoreSkippedSpecs: true,                    
  reportOnlyFailedSpecs: false,                
  captureOnlyFailedSpecs: false                
});                                            
jasmine.getEnv().addReporter(HtmlReporter);    

Environment: Protractor v3.3.0 Jasmine2 v2.4.1 Node v6.3.0 npm v3.10.5 protractor-jasmine2-screenshot-reporter v0.3.2

Steps to Reproduce: 1) install and configure protractor-jasmine2-screenshot-reporter 2) in protractor.conf.js, set the ignoreSkippedSpecs flag to "true" for this plugin 3) in your spec files, mark one or more tests as focus tests by prepending "f" to an "it" or "describe" block. ("fit" or "fdescribe") 4) run protractor 5) after execution is complete, check the screenshots and report generated by this reporter.

Expected: It would be nice if the "ignoreSkippedSpecs" option tracked which test cases Protractor is actually executing at runtime, so that using either "xit" (to exclude one test) or "fit" (to exclude EVERYTHING BUT one test) would result in only the tests executed being run.

Perhaps, if necessary, a separate flag could be made to track use of focus tests ("fit") and leave the ignoreSkippedSpecs flag as-is.

millerick commented 7 years ago

I support this.

agne-a-t commented 7 years ago

Same issue. Any news?