Making addon dummy apps less special

[ef4]

I would like to make dummy apps less of a special case. Right now they have some weird behaviors, like:

• ember-cli-build.js only really applies to the dummy app, but it appears at the root of the addon. This misleads people into thinking they can influence the addon's behavior by putting things in there.
• when building a dummy app, the project.root directory points at the true root of the addon repo, which is not the root of the dummy app itself. There is a Known Hack™️ for reliably getting the real app root like path.join(project.configPath(), '../..'), since configPath does point at the right place for dummy apps. Due to this special case, many addons don't work correctly when you try to consume them inside a dummy app.
• acceptance tests in tests/acceptance implicitly run against the dummy app, which makes them different from what an app developer has come to expect. Now, obviously there is no other place for them to run against, but this is still a bit jarring when first encountered.
• ember-cli/lib/broccoli/ember-addon exists to build dummy apps, not addons (which makes its name confusing), and the only reason it needs to exist at all is that dummy apps are doing all these weird things.
• I would argue dummy is not a great name and we can come up with something more descriptive. Candidates include sample-app or test-harness-app.
• Because the dummy app and the addon share a package.json, it's not possible to test what happens when an app depends on your addon but does not directly depend on your addon's own addon dependencies. In a dummy app, all the addon's dependencies are forced to be direct dependencies of the app too. For example, if you addon uses ember-cli-sass, you can't test what happens when your addon is used in an app that doesn't happen to also use ember-cli-sass.

The general shape of my proposal is:

• move ./tests/dummy to ./sample-app or whatever other name we bikeshed as a good default one.

• the root ember-cli-build.js would change to mostly delegating to ./sample-app/ember-cli-build.js, in order to make it clear that there's no app config that lives at the root level.

• ./sample-app/ember-cli-build.js would import ember-cli/lib/broccoli/ember-app, not ember-cli/lib/broccoli/ember-addon (which would get deprecated)

• dummy apps would become 100% normal ember apps. They should have their own package.json (like ./sample-app/package.json), and you should be able to cd ./sample-app && ember s and everything works exactly as it would if you were in a normal standalone app. They can have the stock app .eslintrc.js in their own directory, meaning no special lint rules are required.

• our previous work on fixing node_modules resolution means that a normal ember app located in ./sample-app is still able to access all the dependencies in ./node_modules, so no special casing is required there. This also allows multiple dummy apps to choose to include only subsets of their parent addon's devDependencies just by not including them in the dummy app's package.json (this is already how ember-cli works -- no special cases for dummy apps).

• acceptance tests should live inside the app that they are testing. So today's ./tests/acceptance would move to ./sample-app/tests/acceptance. This makes them conform exactly to what app developers already know about acceptance tests.

• ./tests/unit and ./tests/integration can stay where they are, those test suites can (in principle) run even if you choose to have no dummy app at all.

The only chunk of API design that I see missing from the above is deciding in detail what to put into the top-level ember-cli-build such that you can conveniently

• run one or more dummy app's automatically in a single ember s process.
• run a test suite composed of the addon's own integration & unit tests
• run the above test suite and also run the tests in each of any number of dummy apps

A totally strawman example of the new top-level ember-cli-build.js would look like:

const AddonDevSetup = require('ember-cli/addon-dev-setup');
module.exports = new AddonDevSetup({
  sampleApps: [
    {
      // the app we're loading
      app: require('./sample-app/ember-cli-build.js'),

      // where to mount the app in the ember development server. Could 
      // potentially allow seting to `null` if you don't want the app to start 
      // automatically when you do a top-level `ember s`.
      rootURL: '/'

      // whether to run this app's `ember test` when we do a top-level `ember test`
      includeTests: true
    }
  ]
});

[kellyselden]

The first point has a PR https://github.com/ember-cli/ember-cli/pull/7602

[knownasilya]

Should these be split up and submitted as RFCs?

[lennyburdette]

dummy apps would become 100% normal ember apps. They should have their own package.json (like ./sample-app/package.json) ...

• Would this make developers think they can ember install or npm install --save-dev inside the ./sample-app directory?
• Would we want some validation to ensure that ./sample-app/package.json doesn't include any packages not in the addon's package.json? Or would something like ember-cli-dependency-checker handle that for us?
• Does it make sense to use * for versions in the sample-app's package.json to suggest that the sample-app inherits its dependencies from the addon?

  "devDependencies": {
    "ember-cli": "*",
    "ember-source": "*"
  }

Otherwise, :+1: ! I'm hoping that moving acceptance tests into the dummy app make engine tests easier. The sample-app application tests can run setApplication(), while the engine's rendering/container tests use setResolver(engineResolverFor(...)). Though I have no idea how that would work at runtime since setApplication and setResolver set global state ...

[ef4]

Based on what I described, it would necessarily be possible to add dependencies directly to the dummy app if you really want to. I'm not sure we'd want to rule that out -- maybe you need to test your addon both with and without another dependency present. But if you do that, you will need to run npm install in the dummy app.

Sticking to a subset of the main package.json and using * does seem like the sensible default behavior we should encourage.

[ef4]

I edited the main post above to add another bad case I had forgotten about:

Because the dummy app and the addon share a package.json, it's not possible to test what happens when an app depends on your addon but does not directly depend on your addon's own addon dependencies. In a dummy app, all the addon's dependencies are forced to be direct dependencies of the app too. For example, if you addon uses ember-cli-sass, you can't test what happens when your addon is used in an app that doesn't happen to also use ember-cli-sass.

[bravo-kernel]

From an end-user perspective I would really welcome the ability to create multiple dummy/sample apps.

[ASH-Michael]

@bravo-kernel Isn't this already possible with routes in the dummy app?

From an end-user perspective I would really welcome the ability to create multiple dummy/sample apps.

[bravo-kernel]

Not completely separated as far as I know. I think I need something like https://github.com/mansona/testing-mu to get what I'm after; completely isolating ember-addon-docs and my demo-app. Update: but I'm pretty new to Ember so I could be mistaken.

[ASH-Michael]

@bravo-kernel I see what you are trying to accomplish now. Interesting.

[rwjblue]

We are working on closing the ember-cli/rfcs repo in favor of using a single central RFC's repo for everything. This was laid out in https://emberjs.github.io/rfcs/0300-rfc-process-update.html.

Sorry for the troubles, but would you mind reviewing to see if this is still something we need, and if so migrating this over to emberjs/rfcs?