ember-cli / ember-exam

Run your tests with randomization, splitting, and parallelization for beautiful tests.
MIT License
286 stars 62 forks source link

Expose public interface for ember-exam params? #104

Closed fotinakis closed 7 years ago

fotinakis commented 7 years ago

Hey Trent,

Working on getting ember-percy to automatically discover some ember-exam settings, in order to fix parallelization bugs when using both addons together.

The bug is basically that we rely on Testem.afterTests to be called only once, to know when the tests are done. This is of course called multiple times in ember-exam land, so the Percy builds are prematurely finalized when the first partition finishes and hence miss data each build.

Our options are: 1) Discover the ember-exam split number, so we count up and only finalize on the last one. I think we can do that via the -private getUrlParams helper:

import getUrlParams from 'ember-exam/test-support/-private/get-url-params';
getUrlParams()

...but of course this is private. Would you be comfortable exposing a public interface for something like this?

Alternatively:

2) if ember-exam exposed its own "afterAllPartitions" hook to know when all of the test partitions are done, we could use that if we see that ember exam is enabled.

Curious about your thoughts. Thanks for this great addon!

trentmwillis commented 7 years ago

The getUrlParams utility doesn't do anything fancy, it simply parses the URL params from window.location. Since this is just an implementation detail (e.g., how Ember Exam reads the URL param), I wouldn't want to expose it publicly.

You could use any utility that parses URL params to access them yourself.You could even use QUnit.urlParams; the only reason this addon doesn't use QUnit.urlParams is that it is not set when we access the URL.

Currently, the URL params that are unique to ember-exam are _split and _partition. While prefixed with underscores, these aren't actually private, they simply use _ to lessen the chance of collision. I treat them as part of the public API (though they aren't documented).

fotinakis commented 7 years ago

Sounds good @trentmwillis — we'll consider the query parameters the public interface for now. I was thinking if you were to expose a public interface it would be something more generic like getConfig().partition that wouldn't expose that ember-exam pulls them from url params.

This isn't blocking us directly right now so I'll close this.

trentmwillis commented 7 years ago

Good suggestion, I'll consider it and let ya know if I wind up adding something like that.