Meteor build time/refresh time after file save is VERY slow

[cdcv]

_4 Upvotes_ I have reasonably large app that I'm working on. When I make any change to the code and press 'save', the time to build the app and refresh the browser is very long, and also highly variable. It is typically 30-60s. A colleague is working on the same app, and is using a similar machine (both are 1-2 year old high-end MacBook Pro), and in his environment the build time is typically <5s. When I make changes to a small demo project, the refresh time is <2-3s. I have tried a bunch of things, including reinstalling meteor, but to no avail. This is bringing development to its knees. What can I do?

Thank you for your help?

[ekatek]

Hm. That's kind of whacky (we do have an Isobuild:Performance label for just this sort of issues!). Unfortunately, it is very hard for us to debug this without a reproduction. Do you have any idea what sort of complexity could be causing this -- how is your app structured (lots of packages, for example)? How large of an app are we talking about? What are example of file changes that do or don't cause this/is there a pattern?

I'll leave this issue open -- maybe someone actively working on the Isobuild:Performance project will come up with more theories; or we will fix this in one of our constantly-upcoming refactorings. Unfortunately, right now, there is not much that we can do directly without more information.

[cdcv]

Thanks for your message. I've been trying to figure out how to get to the bottom of this, and haven't had much success. The biggest reason is that there seems to be great variability in the refresh times. For example, sometimes I set the code one way to test it, get a 'good' refresh time like 15s, then change it again and get 20s, and then get about 5s more for each additional time that I make a change and hit save. This kind of thing makes it more or less impossible to try to figure out what in the code is causing the issues. There does seem to be an important aspect related to system resources. When I clear out other running programs on my Mac, things can get much better (but still not very good). I've tried removing different parts of the app, and while it certainly gets fast if I cut out most of it, I haven't been able to succeed in isolating an issue. The app uses a few dozen packages. Not sure how you'd like me to quantify how 'large' the app is. Pretty big...?

Sure would be good to be able to fix this. Other people haven't been experiencing slow refresh times? Thanks for your help.

[rclai]

One thing that might help that has helped us with our giant app which also takes forever to reload is to short circuit the hot code reload by clicking the address bar in the browser and hit enter. Do not hit F5 or the browser's reload button.

[pierreozoux]

I'm dying also under refresh time, the same, like between 30-60s :/

The code is here if you want to check (I'm quiet new to meteor, and I know there are rooms for improvement): https://github.com/pierreozoux/TioApp

I found a quiet good solution:

meteor remove sanjo:jasmine velocity:html-reporter velocity:console-reporter

The drawback is that you don't have tests anymore, does it help on your side @cdcv ?

[csto]

I feel your pain @pierreozoux . My app takes 30-45 seconds to refresh and I would consider it a small app. This does not bode well for Meteor. I have tried the app on multiple high end macbooks and they all take about the same amount of time.

[Slava]

We are aware of this problem and working on the improvements. In the meanwhile, I must mention that packages get better caching as of today. So if you split up your app into packages, the rebuild time should be much better.

[mitar]

Are there any recent improvements in build times?

[pajooh]

I'm facing the same issue, my app initially forked from Telescope, the build/refresh time is very annoying

[robertpitt]

Have you tried running with profiling enabled, to help spot where the bulk of the time is being used up.

METEOR_PROFILE=1 meteor

regards.

[pajooh]

@robertpitt thank you. i was not aware of this option. however it seems that it can not help the profiling is ending before the actual wasting time is started. i.e. when the profiler prints its output and the Client modified -- refreshing is printed, the browser takes much time after that to load the new app.

[robertpitt]

Sounds like it's Velocity related to me, Ive tried your app on my i3/4GB laptop and get < 1 second to load the client up after a build..

Give this a try and see if you can see anything funky:

VELOCITY_DEBUG=1 VELOCITY_DEBUG_MIRROR=1 METEOR_PROFILE=1 meteor

[pajooh]

here is the command output from my i5/4GB laptop:

| Select Package Versions: 2563.9
|     new CS.Input: 5.1
|     Input#loadFromCatalog (sqlite): 1606.3
|     check for previous versions in catalog: 0.4
|     new CS.Solver: 53.9
|         Solver#analyze: 53.8
|             analyze allowed versions: 6.0
|             analyze root dependencies: 0.8
|             analyze reachability: 6.2
|             analyze constraints: 39.8
|             analyze pre-releases: 0.9
|             other Solver#analyze: 0.2
|         other new CS.Solver: 0.1
|     Solver#getAnswer: 898.1
|         new Logic.Solver (MiniSat start-up): 3.9
|         require root dependencies: 1.2
|         generate package variables: 229.2
|         generate dependency requirements: 94.5
|         generate constraints: 151.4
|         pre-solve: 202.0
|         forbid packages with no matching versions: 0.0
|         minimize unknown_packages: 0.8
|         minimize conflicts: 92.6
|         minimize unanticipated_prereleases: 0.6
|         calculate previous_root distance costs: 4.2
|         add terms to previous_root_incompat: 0.0
|         minimize previous_root_incompat: 22.0
|         calculate update version costs: 0.0
|         minimize update_major: 0.0
|         minimize update_minor: 0.0
|         minimize update_patch: 0.0
|         minimize update_rest: 0.0
|         minimize previous_root_major: 0.5
|         minimize previous_root_minor: 1.6
|         minimize previous_root_patch: 1.6
|         minimize previous_root_rest: 0.1
|         calculate previous_indirect distance costs: 5.0
|         minimize previous_indirect_incompat: 9.4
|         minimize previous_indirect_major: 0.9
|         minimize previous_indirect_minor: 10.2
|         minimize previous_indirect_patch: 3.5
|         minimize previous_indirect_rest: 0.5
|         calculate new_root version costs: 0.0
|         minimize new_root_major: 0.0
|         minimize new_root_minor: 0.0
|         minimize new_root_patch: 0.0
|         minimize new_root_rest: 0.0
|         lock down important versions: 8.3
|         calculate new_indirect version costs: 0.2
|         minimize new_indirect_major: 0.0
|         minimize new_indirect_minor: 0.2
|         minimize new_indirect_patch: 0.1
|         minimize new_indirect_rest: 0.0
|         minimize total_packages: 48.8
|         generate version map: 1.1
|         other Solver#getAnswer: 3.6
|     other Select Package Versions: 0.2
| 
| Input#loadFromCatalog (sqlite): 1606.3
| generate package variables: 229.2
| pre-solve: 202.0
| generate constraints: 151.4
| generate dependency requirements: 94.5
| minimize conflicts: 92.6
| minimize total_packages: 48.8
| analyze constraints: 39.8
| minimize previous_root_incompat: 22.0
| minimize previous_indirect_minor: 10.2
| minimize previous_indirect_incompat: 9.4
| lock down important versions: 8.3
| analyze reachability: 6.2
| analyze allowed versions: 6.0
| new CS.Input: 5.1
| calculate previous_indirect distance costs: 5.0
| calculate previous_root distance costs: 4.2
| new Logic.Solver (MiniSat start-up): 3.9
| other Solver#getAnswer: 3.6
| minimize previous_indirect_patch: 3.5
| minimize previous_root_patch: 1.6
| minimize previous_root_minor: 1.6
| require root dependencies: 1.2
| generate version map: 1.1
| minimize previous_indirect_major: 0.9
| analyze pre-releases: 0.9
| minimize unknown_packages: 0.8
| analyze root dependencies: 0.8
| minimize unanticipated_prereleases: 0.6
| minimize previous_indirect_rest: 0.5
| minimize previous_root_major: 0.5
| check for previous versions in catalog: 0.4
| other Select Package Versions: 0.2
| calculate new_indirect version costs: 0.2
| other Solver#analyze: 0.2
| minimize new_indirect_minor: 0.2
| minimize new_indirect_patch: 0.1
| other new CS.Solver: 0.1
| minimize previous_root_rest: 0.1
| minimize new_indirect_rest: 0.0
| minimize new_indirect_major: 0.0
| forbid packages with no matching versions: 0.0
| minimize update_major: 0.0
| minimize new_root_major: 0.0
| minimize update_minor: 0.0
| minimize update_patch: 0.0
| minimize update_rest: 0.0
| minimize new_root_rest: 0.0
| minimize new_root_minor: 0.0
| minimize new_root_patch: 0.0
| add terms to previous_root_incompat: 0.0
| calculate update version costs: 0.0
| calculate new_root version costs: 0.0
| Total: 2563.9
| 
| files.readFile: 0.4
| files.exists: 0.1
| Rebuild App: 4623.1
|     Isopack#getUnibuildAtArch: 1.5
|     Isopack#_ensurePluginsInitialized: 0.0
|     files.readdir: 4.9
|     files.stat: 26.5
|     files.realpath: 3.3
|     files.readFile: 35.8
|     bundler.bundle..makeClientTarget: 3715.2
|         Target#make: 3715.1
|             Isopack#getUnibuildAtArch: 14.5
|             Unibuild#getResources: 217.3
|                 Isopack#getUnibuildAtArch: 18.9
|                 other Unibuild#getResources: 198.4
|             ClientTarget#mergeCss: 3398.2
|             other Target#make: 85.0
|         other bundler.bundle..makeClientTarget: 0.2
|     bundler..writeTargetToPath: 271.9
|         files.rm_recursive: 0.3
|         files.stat: 0.7
|         files.mkdir: 0.2
|         ClientTarget#write: 243.6
|             Builder#reserve: 10.1
|                 files.mkdir: 2.6
|                 other Builder#reserve: 7.4
|             Builder#writeToGeneratedFilename: 48.8
|                 Builder#generateFilename: 9.6
|                     Builder#_sanitize: 4.7
|                     Builder#reserve: 2.8
|                     other Builder#generateFilename: 2.1
|                 Builder#write: 37.2
|                     Builder#_ensureDirectory: 1.3
|                     files.writeFile: 31.4
|                     other Builder#write: 4.5
|                 other Builder#writeToGeneratedFilename: 2.0
|             bundler..writeFile: 131.8
|                 Builder#write: 129.5
|                     Builder#_ensureDirectory: 5.5
|                     files.writeFile: 109.6
|                     other Builder#write: 14.4
|                 other bundler..writeFile: 2.2
|             Builder#writeJson: 5.3
|                 Builder#_ensureDirectory: 0.0
|                 files.writeFile: 0.5
|                 other Builder#writeJson: 4.8
|             other ClientTarget#write: 47.7
|         Builder#complete: 26.7
|             files.rename: 0.3
|             files.rm_recursive: 26.4
|             other Builder#complete: 0.1
|         other bundler..writeTargetToPath: 0.4
|     other Rebuild App: 564.0
| 
| ClientTarget#mergeCss: 3398.2
| other Rebuild App: 564.0
| other Unibuild#getResources: 198.4
| files.writeFile: 141.4
| other Target#make: 85.0
| other ClientTarget#write: 47.7
| files.readFile: 36.3
| Isopack#getUnibuildAtArch: 34.9
| files.stat: 27.1
| files.rm_recursive: 26.7
| other Builder#write: 18.9
| other Builder#reserve: 7.4
| Builder#_ensureDirectory: 6.9
| files.readdir: 4.9
| other Builder#writeJson: 4.8
| Builder#_sanitize: 4.7
| files.realpath: 3.3
| files.mkdir: 2.9
| Builder#reserve: 2.8
| other bundler..writeFile: 2.2
| other Builder#generateFilename: 2.1
| other Builder#writeToGeneratedFilename: 2.0
| other bundler..writeTargetToPath: 0.4
| files.rename: 0.3
| other bundler.bundle..makeClientTarget: 0.2
| files.exists: 0.1
| other Builder#complete: 0.1
| Isopack#_ensurePluginsInitialized: 0.0
| Total: 4623.6
=> Client modified -- refreshing

as i said, the major reload time(~30seconds) occures after Client modified -- refreshing

[robertpitt]

Although I would love to help, I think we are filling the issue tracker up with client related performance issues, not specifically a problem with meteor but an edge case based on your hardware and a specific set of code.

As this is specifically happening on your machine and the application runs very fast for me and your colleagues, you will just have to dig further and use StackOverflow as a resource for this one.

Sorry I can't help. Good Luck.

[ORESoftware]

hello people! doesn't Meteor have LiveReload / Hot reloading / Hot Code Push (HCP)? This would de-necessitate a full rebuild. Or am I mistaking what HCP is? Seems like HCP meaning has been changed from hot reload to push changes to mobile devices.

http://info.meteor.com/blog/hot-code-pushes

https://github.com/NucleusIDE/meteor-live-update

SO CONFUSED RIGHT NOW

[dud3]

I thought I was the only one having this issue, rebuilding it takes ages, meh ;(.

[mxs]

With react there is https://gaearon.github.io/react-hot-loader/ which hotloads and keeps state in about 1 to 2 secs.

For those that are using ReactJs with Meteor there is: https://github.com/jedwards1211/meteor-webpack-react/ Which I am keen to try out. Initial look seems to provide the best of both worlds.

[ORESoftware]

Hot reloading with Meteor should be an option so you don't have restart the server - but I can't get any information as to whether this is currently a working feature. Hot reloading with React will not work with Meteor, since Meteor is so different. There is no way it will work, since react-hot-loader works with Webpack, not Meteor's build system.

[mxs]

Apologies, the second link is incorrect from above. Should be https://github.com/jedwards1211/meteor-webpack-react/

Which uses Webpack to package instead of Meteor. It seems a little bit hacky with symlinks but there are quite a few huge benefits such as Hot loading for React components and using npm directly.

[thesunny]

This is a far superior solution to meteor-webpack-react. Easier to understand and relatively easy to setup, even on an existing project. This came out very recently and I'm a fan already.

https://github.com/thereactivestack/meteor-webpack-react-kickstart

I had to switch to this as waiting for rebuilds was too frustrating for me. Unfortunately, this doesn't solve the build time issues on the server. I really wish we could do hot loading on the server using something like a server side version of WebPack.

The downside is it's a little bit harder to get started but as soon as your project is of a decent size, the load times kill you much more than the extra little bit of verbosity.

[ORESoftware]

no offense, but trying to get meteor to work with the webpack build system module loader sounds like a nightmare. the Meteor team should be working to create a "native" solution. Trying to get hot reloading functionality to work with Meteor by incorporating webpack's hot module replacement sounds like a horrible horrible idea where you will spend more time debugging your hot reloading features than coding your app.

I have done a bit of research on hot reloading and here is some information if you are interested: https://medium.com/@the1mills/hot-reloading-with-react-requirejs-7b2aa6cb06e1

I recommend watching Dan Abramov's video before anything

[IstoraMandiri]

Interesting article @ORESoftware but I think you have the wrong impression:

Meteor added hot reloading support as early as 2012, but as far as I can tell it is deprecated, or at least depreciated

This is false; hot-code-push is not deprecated - there's just a performance bug for some people in this issue, probably due to some configuration or hardware quirk.

It is most definitely supported natively, and you should expect to have hot code pushes extremely quickly in most situations. On fresh Meteor projects, for me, they happen in less than a second for CSS updates and within a few seconds for markup/js.

[ORESoftware]

but then I don't understand why build times are such an issue and I also was under the impression that HCP was intended for a different purpose - pushing updates to devices already running your app, and wasn't for development. Basically hot reload for production. I will look for a link to that info. On Sep 18, 2015 3:32 AM, "Chris Hitchcott" notifications@github.com wrote:

Interesting article @ORESoftware https://github.com/ORESoftware but the stuff I think you have the wrong impression:

Meteor added hot reloading support as early as 2012, but as far as I can tell it is deprecated, or at least depreciated

This is false; hot-code-push is not deprecated - there's just a performance bug for some people, probably due to some configuration or hardware quirk.

You should expect to have hot code pushes extremely quickly - on fresh Meteor projects, for me, they happen in less than a second for CSS updates and within a few seconds for markup/js.

— Reply to this email directly or view it on GitHub https://github.com/meteor/meteor/issues/4284#issuecomment-141413777.

[ORESoftware]

see this article on Meteor soft code push vs hard code push

https://meteor.hackpad.com/Hot-Code-Push-design-notes-9o22fy6gruu On Sep 18, 2015 10:25 AM, "Alexander Mills" alex@oresoftware.com wrote:

but then I don't understand why build times are such an issue and I also was under the impression that HCP was intended for a different purpose - pushing updates to devices already running your app, and wasn't for development. Basically hot reload for production. I will look for a link to that info. On Sep 18, 2015 3:32 AM, "Chris Hitchcott" notifications@github.com wrote:

Interesting article @ORESoftware https://github.com/ORESoftware but the stuff I think you have the wrong impression:

Meteor added hot reloading support as early as 2012, but as far as I can tell it is deprecated, or at least depreciated

This is false; hot-code-push is not deprecated - there's just a performance bug for some people, probably due to some configuration or hardware quirk.

You should expect to have hot code pushes extremely quickly - on fresh Meteor projects, for me, they happen in less than a second for CSS updates and within a few seconds for markup/js.

— Reply to this email directly or view it on GitHub https://github.com/meteor/meteor/issues/4284#issuecomment-141413777.

[ORESoftware]

actually this is the better link:

http://info.meteor.com/blog/meteor-hot-code-push

so yeah I can't really find any good information about hot reloading with Meteor, any information on hot reloading as a means to avoid server restarting would be useful to me

[ORESoftware]

IMO, Meteor needs to do a better job making the distinction between Hot Code Push and Hot Reloading, one is for production the other for development, so that's at least one difference

[rclai]

It is important to make the distinction, if it wasn't made already, that the Hot code push of Webpack (React Transform) functions differently from that of Meteor's. Webpack's React transform will update the React components on the fly without tampering with the state and without reloading your browser, whereas Meteor's hot code reload will do a browser reload without tampering with the Session state only.

[ORESoftware]

I see, that's good info thanks

[mxs]

@thesunny thank you for pointing me to https://github.com/thereactivestack/meteor-webpack-react-kickstart. It works wonderfully. :+1: It is implemented much more inline with the 'webpack' and 'Meteor' way compared to the earlier solution I posted.

[thesunny]

@mxs You're welcome. I'd also recommend removing redbox from the webpack config settings. It makes it so that the line numbers are reported incorrectly on the error messages. This wastes time looking for where the actual error occurred. Furthermore, for my app, the redbox error message was covered by some "position: fixed;" divs.

The author has indicated that he won't remove redbox but will try to find a fix for the line numbers.

[dud3]

Any progress on this one ? I ran out of coffee while waiting for it to rebuilt.

[sangyoo91]

... I just stumbled upon the same problem...

I am dying waiting :l

[ORESoftware]

get Meteor to support hot reloading to the fullest

[efatsi]

Followed install instructions and working through the Simple Todo app tutorial is taking 3-4 seconds for rebuilding. Seems pretty rough for such a basic app.

[seanodotcom]

Same here. Just halfway through the TODO tutorial, and it's taking over 10 seconds to rebuild after every save. On Win 8.1 Pro, with a 3.7Ghz Xeon, 256GB SSD, and 16GB of RAM.

[NazarK]

Not exactly a solution: Some small client functions (or collection helpers for example) can be copied from editor and pasted directly to browser console (without saving file to disk to avoid meteor rebuild).

I think should be possible to build some meteor plugin that will reload just one file if it has some specific name.

[quape]

Same for me, small app but 25 secs waiting time after every file save :-( This issue should be top priority. It's not possible to work like this.

[pyoungerv]

agreed. any updates?

[ORESoftware]

just a note: if you so happen to be using a machine with a hdd instead of a solid state drive you will mostly experience 3x- 5x build times On Oct 19, 2015 8:59 AM, "pyoungerv" notifications@github.com wrote:

agreed. any updates?

— Reply to this email directly or view it on GitHub https://github.com/meteor/meteor/issues/4284#issuecomment-149257461.

[zaklampert]

+1. Also having long (45s) reload times on our react meteor app.

EDIT: Just tried @rclai 's approach (short circuiting rather than waiting for the browser to refresh on it's own) and it's actually much faster. Any idea why?

[ghost]

@zaklampert are you using browserify? then that is expected. are you using webpack? I guess not.

On Mon, Oct 19, 2015 at 11:08 PM, zaklampert notifications@github.com wrote:

+1. Also having long (45s) reload times on our react meteor app.

— Reply to this email directly or view it on GitHub https://github.com/meteor/meteor/issues/4284#issuecomment-149347928.

[zaklampert]

@casperanderseneu really, 45s for a reload time is expected? YIkes.

I've started creating a webpack version of our app using starter kit mentioned above. Looks promising.

[rclai]

@zaklampert, I have no clue why reloading the way I described above is faster. I've been using the Webpack boilerplate now though.

[ghost]

@zaklampert Currently thereactivestack/meteor-webpack is the leader of the pack. alternatives previously encountered:

• meteor-rocket/webpack and module
• jedwards1211/webpack

over and out

@stubailo why the react in meteor guide still suggests browserify is beyond me.

On Mon, Oct 19, 2015 at 11:46 PM, Richard Lai notifications@github.com wrote:

@zaklampert https://github.com/zaklampert, I have no clue why reloading the way I described above is faster.

— Reply to this email directly or view it on GitHub https://github.com/meteor/meteor/issues/4284#issuecomment-149356637.

[stubailo]

@casperanderseneu what would that guide look like with webpack?

[ghost]

@stubailo webpack, that's the question. let's find out.

[quangv]

just had to +1, hopefully problem gets fixed...

[NazarK]

Isn't meteor 1.2 solving this problem? From release notes: We've also got a new build pipeline in Meteor 1.2 that rebuilds much faster (especially on large code bases),

[quangv]

so I fixed it for myself.

It might sound too obvious, but for me, I did a "Repair Disk Permissions", also "Repair Disk".

My Mac is overall quicker now too... :)

exact steps I did.

1. downgrade meteor meteor update --release 1.2.0.1
2. update meteor back to current meteor update --release 1.2.0.2
3. Perform a "Repair Disk Permissions" in "Disk Utility"
4. Rebooted into recover mode Reboot+"Cmd+R"
5. Selected my overall HD, and did "Repair Disk"
6. Did another "Repair Disk Permissions" for good measure...

Restarted... BOOM meter autorefresh, hot-loading or whatever, is wicked faster.

Went from 15-20 seconds for a trivial change, down to 1-2 seconds. Grease Lightning.

Node: this is for an app, no bigger than simple-todos

I want to go drink some :beer: now.

keep up good work guys. :+1:

edit: fyi the bigger the app, the slower it gets... I definitely thinking about getting Solid-State drives...

[chrishowes]

Hey guys, just added the PR above that may help speed things up - would love your help w testing / feedback.

[bmustata]

+1