WIP: Fix runFor

[olivercoad]

Fixes #505 This is what I had in mind to make runFor more reliable. It still clones the suites for each browser so that the context is still specific to the browser, but does not add an extra suite to do the browser switching so that running suites in different order doesn't break it.

Types of changes

What types of changes does your code introduce to canopy? Put an x in the boxes that apply

• [x] Bugfix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• [ ] Build and tests pass locally
• [ ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
• [x] I have added necessary documentation (if appropriate)

Further comments

Unfortunately it still doesn't satisfactorily show an error if the browser fails to start. If an exception occurs when starting a browser, then failSuite is run on all suites for that browser instead of running the tests.

But failSuite runs fail with autoFail set to true, just skipping all the tests and nothing actually fails. What is the point of autoFail? Isn't the whole point of failSuite to report all the tests as failed (it's not skipSuite...) https://github.com/lefthandedgoat/canopy/issues/178?

I'm hesitant to just just remove autoFail altogether because I don't understand why it was added.

[lefthandedgoat]

Its going to take me a while to get to pulling this. I bought a house and it is consuming all of my time! Thanks for the PR!

[olivercoad]

No worries. Congratulations on your house!

[siwatanejo]

ping :)

[olivercoad]

@lefthandedgoat any comments on the autoFail thing? This was marked as WIP because without addressing that, it can still pass CI even if browsers fail to open. So #505 should still be open.

[lefthandedgoat]

@olivercoad yes I will look at it before I do a build/release. Do you know how to reproduce having a browser not start?