New API to test different ember-cli versions

[kellyselden]

This line determines the ember-cli version used to test your addon. It would be nice if we can come up with an API that allows us to generate a couple "pristine" seed apps with different ember-cli versions.

[danmcclain]

It actually uses the version defined in your addons package.json

[danmcclain]

This line here is the one that updates bower file https://github.com/tomdale/ember-cli-addon-tests/blob/master/lib/utilities/pristine.js#L112

[kellyselden]

@danmcclain Looks like you're thinking of the ember version, not the ember-cli version.

[danmcclain]

Ah, doh, I misread, but we could replicate that logic (or the ember data logic here: https://github.com/tomdale/ember-cli-addon-tests/blob/master/lib/utilities/pristine.js#L229)

If I remember correctly, it generates a new app with the local to the addon under test's node_modules folder, not the one defined for this addon. I'm on mobile right now, will try to follow up shortly

[kellyselden]

I don't think you can use that method, because the pristine app has already generated using the ember-cli version at that point. I'm still pretty sure you only get the ember-cli version from https://github.com/tomdale/ember-cli-addon-tests/blob/3e5cbc6ee2ed7d9607d87695cf25119845661246/package.json#L23

I guess you could change the ember-cli version like you do the ember and ember-data version, then re init and install the new deps, but that would take forever.

[danmcclain]

It's using the node_modules of the addon you are developing, it calls findup from the node_modules/ember-cli-addon-tests https://github.com/tomdale/ember-cli-addon-tests/blob/master/lib/utilities/pristine.js#L325

[danmcclain]

That code also makes it so you can't npm link ember-cli-addon-tests

[kellyselden]

hmm maybe a number of things I was doing (linking, windows, using new code from my pull requests) was causing me to use the ember-cli version from this addon. I remember tweaking the version in this package.json and having different results. I guess the plot thickens.

[danmcclain]

It took me forever to remember where this code was, it's definitely hard to follow without knowing up front, and the assumption you made was the exact same one I made at the time. Better documentation is required 😄

[kellyselden]

But regardless of the source of the ember-cli executable, would you agree that a way to test against any/multiple versions of ember-cli would be beneficial? The addon could be doing broccoli or blueprint stuff, where the version of ember-cli matters.

[danmcclain]

Yes, and we may be able to use ember-try to solve this

[simonihmig]

I think that should not be too hard to accomplish. We would need a separate npm i ember-cli@<version> step to some temporary folder, before everything else starts, and use that folder as our emberCLIPath.

However it should be clever enough to not do this when the ember-cli version is not specified, to reuse the already installed version as it is doing now. This thing is already slow enough, especially on systems with slow disk I/O (no SSD), with the ember new step taking most cycles. Another separate npm i ember-cli with all its deps will probably not help in this regard...

[simonihmig]

That code also makes it so you can't npm link ember-cli-addon-tests

@danmcclain Exactly. Already lost some fair amount of time because of this. Makes testing changes in this addon quite annoying. (Basically had to copy changes into my addonUnderTest/node_modules/ember-cill-addon-tests, any better way?)

Will add an issue...

[stefanpenner]

But regardless of the source of the ember-cli executable, would you agree that a way to test against any/multiple versions of ember-cli would be beneficial? The addon could be doing broccoli or blueprint stuff, where the version of ember-cli matters.

Rather then invoking a subprocess, invoking it in the same process would be nicer. Not only would it make debugging reasonable, but it would also allow easy configuration of which ember-cli version, by allowing it to be passed in via config, thus also allowing ember-try'esq solution to "just work"

[jelhan]

You could fine an example how to run the tests against different Ember CLI versions in TravisCI here: https://github.com/rwjblue/ember-cli-content-security-policy/pull/115