Incredibly slow [initial] build times with ember-cli 0.2.1

[thomasjmwb]

after creating a new ember app I'm getting what I think is very slow build time.. with a new app I see the following:

rebuild:

What should I expect build time for a pretty small app to be? I'm seeing this problem magnified in a much larger app: Some more context around the big app:

Our Brocfile.js / bower.json: https://gist.github.com/thomasjmwb/b4bbdde5e9c1a972de7d

file # stats:

total files: 1045 ./pod-a 41 ./pod-b: 52 ./adapters: 29 ./pod-c 52 ./pod-d ./pod-e 46 ./pod-f 12 ./pod-g 184 ./pod-h 16 ./commons: 16 ./components: 50 ./controllers: 18 ./data-access: 2 ./pod-i 14 ./helpers: 1 ./mixins: 27 ./models: 15 ./pod-k: 64 ./properties: 19 ./routes: 14 ./serializers: 31 ./pod-l: 48 ./styles: 67 ./pod-m: 21 ./templates: 94 ./transforms: 14 ./views: 59

I am on windows, but our developers on mac and linux are both seeing long build times like i've described. I've also tried the prescribed solutions for windows, using ember-cli-windows-addon, making sure i dont have virus scanners running etc.

Could someone point out some potential pitfalls that I'm hitting with a large project?

[stefanpenner]

What should I expect build time for a pretty small app to be? I'm seeing this problem magnified in a much larger app:

You need to consider 2 different build-phases when considering performance. This will help me understand if there is a regression or not.

• initial build (will likely always be slowish)
• incremental builds (should be 800ms -> 3000ms right now)

It seems your large issues are on initial build, that is currently expected since we do a clean build on re-build, to avoid invalid builds. There are plans to introduce a cache, but their exists complexity in knowing when to blow away that cache.

Two slow parts of initial build are:

• source map concat of vendor
• initial babel (es6 -> es3/5) transpilation of your app.

If you do not want to use babel for your app, it is not currently required you can safely remove it from your package.json

Also, it appears you are on windows please be sure to thoroughly review: http://www.ember-cli.com/#windows we have a recent submission from the lovelly Microsoft OSS engineer @felixrieseberg which aims to try and fix-up some windows issues.

[stefanpenner]

the positive thing is, if we introduce a cache for babel (although it is tricky to know when to evict that cache) initial builds will drop by nearly 30s. I know my co-workers would also really appreciate that, as one of our biggest apps is about 70kloc.

[felixrieseberg]

@thomasjmwb: Thanks for reporting. I'd love to see if ember-cli-windows helps with build times. If not, I'm sure that we can improve them.

EDIT: I just saw that you already used ember-cli-windows. This makes things a bit more curious... I'll see what kind of build times I see.

[stefanpenner]

@felixrieseberg i believe incremental are within tolerable levels, I suspect he recently got babel as a dependency and ways surprised by the upfront cost of a non-cached babel build.

[stefanpenner]

I believe the only resolution to this, is enabling a persistent cache for the big offenders, sourceMapConcat and babel. We will have to nail down the heuristics we feel are acceptable for such a cache and implement.

[felixrieseberg]

Here are my numbers:

• Without ember-cli-windows and non-administrative account: ~14000ms initial, ~11000ms incremental build.
• With ember-cli-windows and administrative account: ~4257ms initial, ~1276ms incremental build.

I'm running preview bits of Windows 10 and have full encryption activated, so those numbers should be a bit smaller on a non-encrypted drive with an optimized build. @thomasjmwb: Do me a favor and install the latest version of ember-cli-window, run it again (just to be sure) and serve/build from a PowerShell with administrative rights (Run as 'Administrator').

@stefanpenner: I think the numbers (on my system) are totally fine. Incremental builds of a second are totally within range (and basically identical with my MBP13's build times).

[thomasjmwb]

Thanks for the speedy reply stefan! I have thoroughly reviewed that section and have tried using the ember-cli-windows suggestion. I have even gone so far as to disable the windows search indexing service and windows defender service.

Currently I am running into the ENOTEMPTY error, forcing me to run ember server every time i make a change to a file.

I've tried downgrading ember-cli to 0.1.15, the initial build took the same amount of time, rebuild was about half the time (but still seems very large to me):

I've also just tried building our app fresh on another windows box, including all suggestions in the windows section.

• Ember-cli 0.2.1
• Disabled windows indexing service and windows defender service, made sure those processes were closed
• No IDEs open (like sublime/intelij that would be indexing files)
• make use of ember-cli-windows
• run ember server in cmd that was opened using administrator

This is the best time we are able to get ember server to rebuild in:

edit: cmd, not powershell

[stefanpenner]

Currently I am running into the ENOTEMPTY error, forcing me to run ember server every time i make a change to a file.

so this seems like the root of your/the problem

[thomasjmwb]

it is certainly an issue, but we are seeing similar rebuild times across other environments that do not run into the ENOTEMPTY error. My team member on a OS X is getting rebuild times of 4~6 seconds (awesome!) while my other team member on ubunto 14.04 is getting rebuild times of 19 seconds.

[thomasjmwb]

We've made some improvements and managed to lower the initial build and rebuild time...

When I'm running cmd with admin priv (instead of powershell) I'm managing to avoid the ENOTEMPTY error, but still seeing the following:

[stefanpenner]

@thomasjmwb are you on an SSD or HDD?

[stefanpenner]

those number still seem pretty high.

[jschilli]

We have a pretty large app (2k files, 125k sloc)

On a 2 core i7 macbook (8gb ram - ssd) initial build 105000 ms (details below)

While the initial build time is painful, the rebuild times hovering around 10 sec is reasonable.

We did disable hinting hinting: false in EmberApp options to help drive down initial build time. We also disabled es3Safe although that's probably a no-op with the most recent ember-cli

I've spent a lot of time debugging broccoli trees and looking at addons invalidating trees w/o needing to.

Current pet project: es6 opt-in so power of babel but only as desired/required

Initial build:

version: 0.2.1
Livereload server on port 35729
Serving on http://localhost:4200/

Build successful - 104756ms.

Slowest Trees                  | Total
-------------------------------+----------------
Babel                          | 44985ms
Babel                          | 27504ms
ES6: App Tree                  | 9934ms

Change one .js file:

file changed views/branches.js
file changed views/branches.js

Build successful - 11860ms.

Slowest Trees                  | Total
-------------------------------+----------------
Concat: App                    | 2131ms
ES6: App Tree                  | 1394ms
Babel                          | 1388ms
Class                          | 1327ms
EmberInlineTemplatePrecompiler | 1248ms

After removing babel: (clean rebuild)

unbuild ember-cli-babel@4.3.0
version: 0.2.1
Livereload server on port 35729
Serving on http://localhost:4200/

Build successful - 29009ms.

single js file change w/o babel:

file changed views/branches.js
file changed views/branches.js

Build successful - 9432ms.

Slowest Trees                  | Total
-------------------------------+----------------
Concat: App                    | 2181ms
ES6: App Tree                  | 1381ms
EmberInlineTemplatePrecompiler | 1189ms
Class                          | 1110ms
Funnel: App JS Files           | 510ms

[thomasjmwb]

HDD.

my tmp folder is now at ...5.87Gb. I'm gonna go ahead and restart my computer and see what the build time is then.

Possible unrelated detail i'd like to volunteer in case it happens to be relevant.. we have a couple files that are ~3MB each (very large JSON trees in fixture data). Perhaps these files contribute to such a huge tmp directory?

[stefanpenner]

The huge temp dir is the result of a crashing process or control-c and we may not be doing the correct cleanup?

HDD will likely get a bit faster with some planned work, but that is likely the source of the somewhat slowish builds.

We have some WIP that will reduce the amount of internal work we do dramatically. I suspect it will help the HdD users more then the SSD users.

[thomasjmwb]

@jschilli

I've spent a lot of time debugging broccoli trees and looking at addons invalidating trees w/o needing to.

Could you elaborate a bit more on that? Or just point me to whatever resources you checked out while debugging broccoli trees? I want to start seeing if I can optimize our build process but I'm unsure of where to start since it's hard to tell just from the ember-cli docs what I'm getting out of the box. For example, how I should go about changing how often we recompile test/app/vendor files.

[jschilli]

I don’t have a list of resources, rather hard won insights.

running DEBUG=“*” ember serve provides reams of data from many of the underlying bits. I usually start there and then focus in on a specific set of debug flags to identify issues.

For instance, I was having slower rebuild times and determined that an addon was not caching its’ result and therefore on every rebuild was forcing a significant cache invalidation.

Adding caching in my addon resolved that.

Looking at the sane watcher/slow trees output alone can help focus. We had an issue where saving a .js file was causing the SassCompiler to run - reasoning about that led to an addon which turned into an ember-cli patch for postprocessTree

Finally use of broccoli-stew https://github.com/stefanpenner/broccoli-stew https://github.com/stefanpenner/broccoli-stew - specifically log (and the new afterBuild ) are useful for diagnosing trees/caching issues etc

HTH

On Mar 25, 2015, at 2:17 PM, Thomas Whalen-Bridge notifications@github.com wrote:

@jschilli https://github.com/jschilli I've spent a lot of time debugging broccoli trees and looking at addons invalidating trees w/o needing to.

Could you elaborate a bit more on that? Or just point me to whatever resources you checked out while debugging broccoli trees? I want to start seeing if I can optimize our build process but I'm unsure of where to start since it's hard to tell just from the ember-cli docs what I'm getting out of the box. For example, how I should go about changing how often we recompile test/app/vendor files.

— Reply to this email directly or view it on GitHub https://github.com/ember-cli/ember-cli/issues/3637#issuecomment-86156863.

[thomasjmwb]

Thanks a ton @jschilli !

I'm also checking out https://github.com/ember-cli/ember-cli/blob/master/lib/broccoli/ember-app.js as a good entry point to get an idea of what happens after dropping a bunch of stuff in my app's Brocfile.

[jschilli]

yeah, reading the source and understanding the flow helps a lot in debugging perf issues

On Mar 25, 2015, at 2:30 PM, Thomas Whalen-Bridge notifications@github.com wrote:

Thanks a ton @jschilli https://github.com/jschilli !

I'm also checking out https://github.com/ember-cli/ember-cli/blob/master/lib/broccoli/ember-app.js https://github.com/ember-cli/ember-cli/blob/master/lib/broccoli/ember-app.js as a good entry point to get an idea of what happens after dropping a bunch of stuff in my app's Brocfile.

— Reply to this email directly or view it on GitHub https://github.com/ember-cli/ember-cli/issues/3637#issuecomment-86161320.

[thomasjmwb]

Update--

After making some pretty drastic changes to try and get some reasonable windows performance this is what our brocfile looks like:

/* global require, module */

var EmberApp = require('ember-cli/lib/broccoli/ember-app');
var unwatchedTree    = require('broccoli-unwatched-tree');
var app = new EmberApp({
  storeConfigInMeta: false,
  tests: false,
  trees: {
    'public': unwatchedTree('public'),
    'tests': unwatchedTree('tests')
  },
  es3Safe: false,
  outputPaths: {
     app: {
       js: '/assets/scripts/app.js'
     },

     vendor: {
    css: '/assets/stylesheets/vendor.css',
      js: '/assets/scripts/vendor.js'
     }
  },
  stylusOptions: {
    outputFile: '/stylesheets/app.css'
  },
  gzip: {
    keepUncompressed: true
  },
  fingerprint: {
    enabled: false
  }
});
// along with the exports of each module as its value.
app.import('bower_components/simpleStorage/simpleStorage.js');
app.import('bower_components/moment/moment.js');
app.import('bower_components/d3/d3.js');
app.import('bower_components/ember/ember-template-compiler.js');
app.import('bower_components/lodash/lodash.js');
app.import('bower_components/typeahead.js/dist/bloodhound.js');
app.import('bower_components/typeahead.js/dist/typeahead.jquery.js');
app.import('bower_components/zeroclipboard/dist/ZeroClipboard.js');
app.import('bower_components/bootstrap/dist/js/bootstrap.min.js');
app.import('bower_components/bootstrap/dist/css/bootstrap.min.css');
app.import('bower_components/jquery-ui/ui/jquery-ui.js');
//required for resonate template naming pattern: http://rlbaxter.us/posts/#!post/extend-the-ember-resolver
app._podTemplatePatterns = function() {
    return this.registry.extensionsForType('template').map(function(extension) {
        return new RegExp('\.(.+\.)?' + extension + '$');
    });
};

module.exports = app.toTree();

windows performance (HDD):

os x performance (SSD):

ubuntu performance (HDD):

Pretty drastic differences, all running the exact same code. The resident windows guru on our team seems to think the primary cause of the slow windows performance is the heavy I/O occurring during rebuild. I'm not really sure what other information to provide, please let me know if theres anything else that someone might find useful.

[thomasjmwb]

Some additional stats on windows ember serve with DEBUG="*":

file changed account\routes\account-list.js
Wed, 25 Mar 2015 23:35:38 GMT broccoli-sane-watcher scheduleBuild: C:\directory-to-my-app\app\account\routes\account-list.js
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher build: C:\directory-to-my-app\app\account\routes\account-list.js
Wed, 25 Mar 2015 23:35:39 GMT ember-cli:utilities/attempt-metadata-index-file not darwin, skipping tmp/.metadata_never_index (which hints to spotlight to prevent indexing)
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) app
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) config
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/loader.js
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/jquery/dist
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/handlebars
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/ember
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/ember-cli-shims
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/ember-resolver/dist/modules
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/ember-load-initializers
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/ember-data
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/simpleStorage
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/moment
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/d3
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/lodash
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/typeahead.js/dist
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/zeroclipboard/dist
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/bootstrap/dist/js
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/bootstrap/dist/css
Wed, 25 Mar 2015 23:35:39 GMT broccoli-sane-watcher addWatchDir: (not added duplicate) bower_components/jquery-ui/ui
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _validation
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer validation.react
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer spec.functionName
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer spec.blockScopedFunctions
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer flow
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _modules
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer es6.tailCall
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _blockHoist
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _declarations
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _aliasFunctions
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _strict
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _moduleFormatter
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer es3.propertyLiterals
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer es3.memberExpressionLiterals
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _cleanUp
Wed, 25 Mar 2015 23:35:41 GMT babel my-app/account/routes/account-list.js: Running transformer _declarations
Wed, 25 Mar 2015 23:35:54 GMT broccoli-sane-watcher triggerChange
Wed, 25 Mar 2015 23:35:54 GMT ember-cli:watcher didChange { directory: 'C:\\directory-to\\my-app\\tmp\\tree_merger-tmp_dest_dir-UijLU0bs.tmp', graph: Node:173 subtrees: 5 selfTime: 73048477 totalTime: 12620836792, totalTime: 12620836792, filePath: 'C:\\directory-to\\my-app\\app\\account\\routes\\account-list.js' }

Build successful - 12621ms.

Slowest Trees                  | Total
-------------------------------+----------------
Funnel: App JS Files           | 3089ms
ES6: App Tree                  | 2010ms
Babel                          | 1297ms
Funnel: Filtered App           | 912ms
Concat: App                    | 839ms
Concat: Vendor                 | 768ms

[jschilli]

@thomasjmwb impressive gains. heavy i/o and HDD is almost certainly contributing to slow(er) build times on windows.

One thing that I noticed is 'tests': unwatchedTree('tests') will be a buzz kill for devs expecting to TDD

I have on my list to look at the addWatchDir which happens on every rebuild - the sum of those actions runs to about 900ms on my builds - because I don't yet know what it's doing I can't comment

@thomasjmwb are you using es6 features? have you tried running without babel?

[thomasjmwb]

We are only using es6 module syntax at this time. We considered ditching babel, but our estimated gains (based on the Slowest Trees table logged out at the end of a build) did not make it seem like it was worth the effort considering we have ~1000 files to futz with.

@ tests, agreed, it is definitely not ideal.

we are planning on doing something where you add '-e tests' to ember serve to enable tests so developers can go that route at their discretion

[bpcrao]

I am facing the same issue

[stefanpenner]

@bpcrao it looks like you may not have followed the instructions @ http://www.ember-cli.com/#windows although maybe you have and are just being trolled by a combination of not having an HDD, being on windows, a bug introduced when source maps where added (causing more things to be rebuilt then expected).

[bpcrao]

@stefanpenner I have followed all of those

• disabled windows defender and Indexer
• enabled symlinks , checked it with can-symlink
• running in administration mode
• tmp folder is clean
• I am using ES6(so babel is needed)
• excluded the project folder from Antivirus too

[stefanpenner]

@bpcrao have you used the microsoft provided add-on ?

npm install --save-dev ember-cli-windows-addon
ember windows

• are you on an HDD?
• do you have an editor watching tmp (like sublime or webstorm etc)

[bpcrao]

@stefanpenner Yes installed the windows addon.

are you on an HDD? Yes do you have an editor watching tmp (like sublime or webstorm etc) - No , excluded from my editor.

bower_components 26 MB node_modules 100 MB tmp 45 MB

Anything else i need to check ? and my brocfile.js is untouched, the one which comes with ember new "appname"

[stefanpenner]

Anything else i need to check ?

• os version?
• number of files in app/*
• number of files in test/*
• number of ember addons

[bpcrao]

• windows 7(Microsoft Windows [Version 6.1.7601]) and on Windows 8
• app/ 923 files and 6 MB
• tests none
• addons: default configuration of 10
• using ember-cli 0.2.3
• Maximum CPU usage of 80%
• RAM Usage of around 10GB

addons. "broccoli-asset-rev": "^2.0.2", "ember-cli": "0.2.3", "ember-cli-app-version": "0.3.3", "ember-cli-babel": "^4.0.0", "ember-cli-content-security-policy": "0.4.0", "ember-cli-delay-app-boot": "1.0.1", "ember-cli-dependency-checker": "0.0.8", "ember-cli-htmlbars": "0.7.4", "ember-cli-ic-ajax": "0.1.1", "ember-cli-inject-live-reload": "^1.3.0", "ember-cli-qunit": "0.3.9", "ember-cli-uglify": "1.0.1", "ember-cli-windows-addon": "^1.2.2", "ember-data": "~1.0.0-beta.15", "ember-export-application-global": "^1.0.2"

[bpcrao]

@stefanpenner

tried with SSD, build and rebuild improved by 50%

build 25 sec rebuild 12 sec

[stefanpenner]

build 25 sec

this is expected, although some work exists (i still need to port it to windows before i can release it) to make warm boots much faster. for reference: https://github.com/stefanpenner/broccoli-persistent-filter

rebuild 12 sec

This is still somewhat expected, we regressed when adding source maps. We are doing work now to once-again reduce this.

I also believe the work to fix the regression we introduced with source maps, will nicely improve the HDD case.

[tdesmet]

I am facing a similar issue although in a bigger proportion

running windows 8.1 Pro and have an ssd. I used the ember-cli-windows command line tool to disable windows defender and windows search for the tmp directory. And I am running the command line as an administrator.

my build times are really huge while on my coworkers mac there are acceptable.

initial build:

rebuild:

And on my coworkers mac initial build:

rebuild:

I used to have timings as my coworker or a bit slower,

The only big change I know of is that we recently upgraded to ember-cli 0.2.3...

[bpcrao]

@tdesmet

are you able to create symlinks ? can you check with npm package can-symlink .

[tdesmet]

@bpcrao yes I am able to build symlinks

I figured out what was going on, because of https://github.com/emberjs/ember.js/issues/10310 we are building our own patched version of ember in the bower_components directory. The result being that the bower_components directory is a lot bigger then it should be because of the npm and bower packages that get installed. After cleaning this up the initial build time is now 46787ms

Still slow imo but much better then before.

[bpcrao]

@tdesmet Even the tmp folder size impacts your build time.

[ghost]

According to this.

Broccoli rebuilds everything over and over again, including things which did not change. To solve this, manually tweak your build process to exclude certain things and do them manually as-needed. For instance, let's say you are bringing in a 200KB external JS package via bower, and app.import'ing it in your Brocfile.js. As a temporary work-around, and to see if it makes a difference, remove that file from the Broccoli world and load it directly in index.html. The same applies to large CSS files.

Can anyone confirm this or point to where I can check this myself in the source code?

[stefanpenner]

Broccoli rebuilds everything over and over again, including things which did not change.

This isn't entirely true, Broccoli is like react.js where it "rebuilds" the world, but unless bugs exists (some still do) only the changes need to be processed.

For windows users, our friends at Microsoft (who use ember and ember-cli) have provided an add-on, to correctly adjust windows defaults to allow for symlinks see: http://www.ember-cli.com/#windows. This ultimately enables same-order performance times to comparable posix systems.

One thing worth noting, recently we had a performance regression in our concat phase. I fixed part of it last weekend, and plan to fix the next chunk this weekend. Hopefully soon that regression will be gone,

For reference, one of our apps at work has > 120,000 loc of just app code, and a huge number of addons.

Without my fixes:

• initial build 80seconds
• incremental build 18seconds

this is extremely poor

With my fixes:

• initial build 15 seconds
• incremental build 2 seconds.

The fixes are a bit tricky, so the work require to push the changes through the whole ecosystem has taken more time the expected. Luckily one of the blockers, the stable broccoli-plugin api landed (last week). Which will help us get the ball moving again :)

---

Sorry, for the current regression of performance, we will sort it out again shortly.

[ghost]

It should rebuild only what's dirty, which is what I believe React does. Isn't this what Broccoli does too?

...unless bugs exists (some still do) only the changes need to be processed.

How far back would one need to downgrade ember-cli in order to note any perf improvements? Even if not a real solution, it would help our team get some relief.

[ghost]

@stefanpenner

One thing worth noting, recently we had a performance regression in our concat phase. I fixed part of it last weekend, and plan to fix the next chunk this weekend. Hopefully soon that regression will be gone,

Do you mind sharing some details on this? Like some of the places where you think bottlenecks could occur or just any general hints as to where start looking?

Thanks.

[stefanpenner]

I'll have to verify my recent tunings have improvement windows performance (i'll likely do so tomorrow).

relevant details

TL;DR turbo button enabled for many previously pathological cases.

given one specifically poor example, on my laptop (OSX SSD)

original times:

20,000ms initial
12,000ms incremental

with fixes times:

3,500ms initial
1,800ms incremental

and 1,600 of the remaining time looks like more low hanging fruit.

[tsing80]

Just tried ember-cli from latest master on windows, it has similiar build time as 1.13.7 for the below testing repostory: https://github.com/tsing80/testperf

Initial build:

ember serve
version: 1.13.7-master-2292948789
bootstrap-sassy config:  all JS enabled, glyphicons enabled
Livereload server on http://localhost:49156
Serving on http://localhost:4200/

Build successful - 21429ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
Concat: Vendor                                | 3012ms
TreeMerger (vendor)                           | 2359ms
Babel                                         | 1361ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
Babel (15)                                    | 5788ms (385 ms)
Concat: Vendor (1)                            | 3012ms
BroccoliMergeTrees (8)                        | 2423ms (302 ms)
TreeMerger (vendor) (1)                       | 2359ms
Funnel (117)                                  | 1523ms (13 ms)

Incremental build:

file changed app.js

Build successful - 11154ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
TreeMerger (vendor)                           | 3008ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
TreeMerger (vendor) (1)                       | 3008ms
Babel (15)                                    | 1136ms (75 ms)
Funnel (117)                                  | 1111ms (9 ms)
BroccoliMergeTrees (8)                        | 721ms (90 ms)
Funnel: Addon JS (16)                         | 632ms (39 ms)

There is 1580 directories in the tmp directories , while there is only around 8XX directories in 1.13.7

[stefanpenner]

@tsing80 it looks like your app doesn't hit any of perf issues the recent PR's have been targetting. (large bower_components/vendor trees);

That being said, it may have not included some of my latest changes to Funnel. (or maybe its slow IO on windows – which i have some ideas for)

It is hitting 2 major areas I hope to land solutions for shortly:

• changing app tree, invalidates vendor and css trees (to improve incremental builds)
• some core filters are non-caching, causing extra invalidations (causing downstream changes for no reason)
• broccoli-filter utilizing persistent caches (to have fast initial builds)

Some quick questions:

• what version of windows
• are symlinks enabled?
• SSD or HDD

example app running on OSX

version: 1.13.7-master-df6348da43
bootstrap-sassy config:  all JS enabled, glyphicons enabled
Livereload server on http://localhost:49152
Serving on http://localhost:4200/

Build successful - 7275ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
Concat: Vendor                                | 2400ms
Babel                                         | 691ms
Babel                                         | 581ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
Babel (15)                                    | 2818ms (187 ms)
Concat: Vendor (1)                            | 2400ms

Job 1, 'ember s ' has stopped
➜  testperf git:(master) ✗ bg
Send job 1 'ember s ' to background
➜  testperf git:(master) ✗ touch app/app.js
➜  testperf git:(master) ✗ file changed app.js

Build successful - 1205ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
TreeMerger (vendor)                           | 216ms
Concat: Vendor                                | 143ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
TreeMerger (vendor) (1)                       | 216ms
Concat: Vendor (1)                            | 143ms
Funnel (117)                                  | 130ms (1 ms)
Babel (15)                                    | 115ms (7 ms)
BroccoliMergeTrees (8)                        | 69ms (8 ms)

➜  testperf git:(master) ✗ touch app/app.js
➜  testperf git:(master) ✗ file changed app.js

Build successful - 1130ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
TreeMerger (vendor)                           | 216ms
Concat: Vendor                                | 142ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
TreeMerger (vendor) (1)                       | 216ms
Concat: Vendor (1)                            | 142ms
Funnel (117)                                  | 117ms (1 ms)
Babel (15)                                    | 92ms (6 ms)
BroccoliMergeTrees (8)                        | 65ms (8 ms)

[stefanpenner]

of note, the original posting issue will likely be dramatically improved by persistent caches in broccoli-filter.

for reference: https://github.com/stefanpenner/broccoli-persistent-filter it is still pending windows support before it can be released.

Some numbers, at work where have been using persistent-filter for babel, which reduces our initial build babel times from 60s -> 2s, a very welcome speedup.

[stefanpenner]

@bpcrao yes I am able to build symlinks

I figured out what was going on, because of emberjs/ember.js#10310 we are building our own patched version of ember in the bower_components directory. The result being that the bower_components directory is a lot bigger then it should be because of the npm and bower packages that get installed. After cleaning this up the initial build time is now 46787ms

@bpcrao I believe you issue will be resolve (or mitigated) as a result of https://github.com/ember-cli/ember-cli/issues/3637#issuecomment-128921482

[stefanpenner]

@tsing80 I have also noticed something, ember-cli-jquery-ui is looping over files in the directories in-order to import them. This is resulting in an unexpectedly high number of app.import, that all operate

A quick glance suggests if I where to add globbing support to app.import that project could batch together its 50+ imports, into 2. Resulting in 50 or so less full dir scans, mergers filters etc and we can likely mitigate the final n funnels to filter the resulting trees, as they will be pre-filtered. Wow, im tired just saying that.

This seems like a great little area to improve

I have opened: https://github.com/ember-cli/ember-cli/issues/4644 to specifically track what i see, and describe a set of proposed solutions

[tsing80]

@stefanpenner

• WIndows 7 is used
• symlink is enabled (ember is run with Administrator right)
• HDD is used, I have tried to use RAM disk also in the tmp directory, the time using ramdisk is around half of the original time

[stefanpenner]

for those interested, come help populate the stress app https://github.com/ember-cli/stress-app/issues/2

[stefanpenner]

upgrading the following libs will help alot:

• ember-cli-babel (just doing a nom)
• ember-cli-qunit to ^1.0.0

This will also come out with the next CLI release by default.

TL;DR your initial build will be slightly slower, but subsequent restarts will have a primed persistent cache to feed off.

To prevent the cache from growing onbounded:

• The cache is stored in system tmp, which will result in your system pruning it (if un-accessed) on restart or after a-few days in inactivity.
• the cache entries are gziped (cutting down the byte size)
• the granularity of the cache is per file, so only deltas are added.

To prevent getting trolled by stale cache:

• we have a pretty aggressive cache key, that takes many things (like dependencies) into account.
• we are going to continue to improve the confidence in this key over time

Is this code super new and scary?

Not really, we have been using it internally for the last month or two without trouble.

[mmahalwy]

@stefanpenner I'd like to reopen this issue. We posted a question here: http://discuss.emberjs.com/t/ember-cli-compilation-time/8834

We have spent quite a bit of time updating all our dependencies to the latest versions in hopes compile time would be faster. We are using 1.13.8.

Our initial load is ~144000ms

Build successful - 144294ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
EslintValidationFilter                        | 55845ms
EslintValidationFilter                        | 17441ms
Babel                                         | 17105ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
EslintValidationFilter (2)                    | 73287ms (36643 ms)
Babel (28)                                    | 30412ms (1086 ms)

And subsequent changes are over 20s.

Build successful - 23611ms.

Slowest Trees                                 | Total
----------------------------------------------+---------------------
Funnel: App JS Files                          | 4796ms
ES6: App Tree                                 | 3656ms
BroccoliMergeTrees                            | 2602ms
SassCompiler                                  | 2184ms
Concat: App                                   | 1821ms

Slowest Trees (cumulative)                    | Total (avg)
----------------------------------------------+---------------------
Funnel: App JS Files (1)                      | 4796ms
ES6: App Tree (1)                             | 3656ms
BroccoliMergeTrees (6)                        | 2635ms (439 ms)
SassCompiler (1)                              | 2184ms
Concat: App (1)                               | 1821ms
TreeMerger (lint) (2)                         | 1589ms (794 ms)
Babel (28)                                    | 1499ms (53 ms)

Our deps:

"dependencies": {
    "@peekops/ember-cli-peek-ui": "1.0.17",
    "active-model-adapter": "2.0.1",
    "babel-eslint": "^4.1.1",
    "broccoli-asset-rev": "^2.0.2",
    "chai": "^3.0.0",
    "dotenv": "^1.1.0",
    "ember-cli": "^1.13.6",
    "ember-cli-accounting": "^1.0.0",
    "ember-cli-app-version": "0.4.0",
    "ember-cli-babel": "^5.1.5",
    "ember-cli-bourbon": "^1.1.0",
    "ember-cli-date-picker": "0.0.13",
    "ember-cli-dependency-checker": "^1.0.0",
    "ember-cli-deploy": "^0.4.1",
    "ember-cli-dotenv": "^0.4.0",
    "ember-cli-emblem": "0.3.0",
    "ember-cli-eslint": "1.2.0",
    "ember-cli-filepicker": "0.1.8",
    "ember-cli-font-awesome": "0.0.9",
    "ember-cli-htmlbars": "1.0.0",
    "ember-cli-htmlbars-inline-precompile": "^0.1.1",
    "ember-cli-ic-ajax": "0.2.1",
    "ember-cli-inject-live-reload": "^1.3.0",
    "ember-cli-locales-pods": "^0.0.9",
    "ember-cli-mocha": "^0.9.2",
    "ember-cli-sass": "^4.2.0",
    "ember-cli-sass-pods": "^1.0.9",
    "ember-cli-styles-reloader": "0.1.6",
    "ember-data": "^1.13.11",
    "ember-deploy-redis": "0.0.5",
    "ember-deploy-s3": "0.0.4",
    "ember-i18n": "^4.1.2",
    "ember-local-storage": "0.0.6",
    "ember-modal-dialog": "0.7.7",
    "ember-responsive": "1.1.0",
    "ember-select-2": "1.3.0",
    "ember-simple-auth": "simplabs/ember-simple-auth#jj-abrams",
    "ember-sinon": "0.2.1",
    "eslint-config-ember": "0.1.0",
    "js-string-escape": "^1.0.0",
    "path": "^0.11.14",
    "sinon": "^1.15.4"
  },
  "devDependencies": {
    "active-model-adapter": "2.0.1",
    "ember-cli-accounting": "1.0.0",
    "ember-cli-deprecation-workflow": "0.1.3",
    "ember-cli-flash": "1.3.3",
    "ember-cli-locales-pods": "0.0.6",
    "ember-cli-mocha": "0.9.2",
    "ember-cli-moment-shim": "0.6.2",
    "ember-cli-release": "0.2.3",
    "ember-cli-simple-auth-token": "mmahalwy/ember-cli-simple-auth-token",
    "ember-cli-sri": "^1.0.1",
    "ember-cli-uglify": "^1.0.1",
    "ember-cpm": "1.3.4",
    "ember-data": "1.13.11",
    "ember-disable-proxy-controllers": "^1.0.1",
    "ember-export-application-global": "^1.0.3",
    "ember-i18n": "4.1.2",
    "ember-moment": "3.6.2",
    "ember-resolver": "2.0.3",
    "ember-watson": "0.6.5"
  }

We cannot find what is taking so much time. We do have a large app (~1000 files) but it's really showing that ember-cli is having a hard time keeping up with large apps like ours making development very difficult and time consuming.

Thoughts?