HTML reporter does not play nice when running in parallel

[chrismikehogan]

When I run N test files in M parallel processes, such that M < N, then report.html will only show the last M test file results.

For example (where N = 3, and M = 2):

    // in protractor.conf.js (partial)
    specs: [
        'spec/one.js',
        'spec/two.js'
        'spec/three.js'
    ],
    capabilities: {
        browserName: 'chrome',
        shardTestFiles: true, // run specs in parallel
        maxInstances: 2 // max number of parallel specs
    },
    //....
    onPrepare(function() {
        jasmine.getEnv().addReporter(new HtmlScreenshotReporter({
            dest: 'report/ptor',
            pathBuilder: function(currentSpec) {
                return 'screenshots/' + currentSpec.fullName;
            }
        }));  
    }),
    // ....

Will result in a report that, assuming two.js and three.js finish after one.js, only shows two.js and three.js results.

This appears to be part of why this happens: https://github.com/mlison/protractor-jasmine2-screenshot-reporter/blob/master/index.js#L246-L256 (note that this code just recently change, though it appears to behave similarly to the fs.unlink that used to be around).

Im not totally certain what the best way to go about fixing this is. Locally, I have simply 86'd the cleanup that is done at start, and have some separate clean up scripts I run when needed, but that might not be the right way.

Should the clean up step linked up be toggleable? Have any other folks encountered this problem and found a more elegant solution?

[AndrewFinlay]

Hi Chris, sorry these changes broke your reporting, I wasn't aware of the sharding interfaces but it looks like something we might be able to use so I'm interested in making a better solution. I have had a look into it and it should be possible to change it a little so that it plays nice with sharded tests. I'll build on your fork since it fixes a few other things. Thanks.

[AndrewFinlay]

Hi Chris, I've built on what you've done and I think I've got a viable solution for sharded test reporting with the set of previous features. It'd be great if I could get your thoughts on it.

[magicode118]

Good job, tried this out and it's working fine. The only issue I have is that showQuickLinks doesn't work with this kind of setup and the failed specs' links are not shown.

[AndrewFinlay]

The quickLinks have all been reimplemented with uuids and the js onload function should collate them in the browser, so they should work as long as all test shards write to the same file.

I forgot to mention that the recent changes to work with async reporting are all running on my fork and not the main repo. To get it to work you'll have to make a few changes to your configuration file too, it needs to do some work in the beforeLaunch and afterLaunch hooks to set up the report file, as these can run synchronously before and after the sharded async tests. I've updated the readme to detail the new basic configuration requirement.

zalow517, could you confirm that you've got the uuids & onLoad function in your report as well as the changes required for the conf file so I know that we're talking about the same build? Thanks.

[mlison]

@AndrewFinlay #40 is looking good imo. @zalow517 @chrismikehogan any comments?

[chrismikehogan]

@AndrewFinlay @mlison Sorry I haven't had a chance to take a close look at 40, and probably wont for a little while.

But no comments from me!

[magicode118]

@AndrewFinlay @mlison #40 is working great. As @AndrewFinlay pointed out, my issue was that of using the wrong fork. Tried it with the following options:

var reporter = new HtmlScreenshotReporter({ cleanDestination: true, showSummary: true, showConfiguration: false, reportTitle: null, showQuickLinks: true, dest: dir, filename: 'report.html', reportOnlyFailedSpecs: false, captureOnlyFailedSpecs: true, });