Increase Build Speed

[englercj]

Right now I have 2 builds (one minified, one not) in my gruntfile. Each task independently takes ~10 seconds to complete. When doing major development where I rebuild multiple times this adds up really fast.

Are there some speed increases coming in the future? Are there any known bottlenecks I could help with? Do I have some kind of configuration issue?

I also notice if I run my default grunt task (which should run both builds) it stops after only running the first build; I think that may be due to some sort of a timeout.

[anodynos]

Are there some speed increases coming in the future?

There is no planned update for a major speed increase - I suppose most of the time is taken by the parsing engine (recently updated from uglify to esprima in v0.6 which is supposed to be faster) & then generating the code from AST with escodegen.

Do I have some kind of configuration issue?

Not an issue really, but the best practice for speed increase while deveoping is to use, grunt-contrib-watch (or plain urequire's -watch) and allow uRequire to rebuild only what has really changed - see the watch section in https://github.com/anodynos/uBerscore/blob/master/Gruntfile.coffee#L388-406 as an example. I use it my self and I easily get almost instant (100-500ms) rebuilds.

if I run my default grunt task (which should run both builds) it stops after only running the first build;

This is very weird & I will look into it ASAP when I get home from work - could you please tell me what environment you are running in (OS/node/grunt/urequire versions) & any messages uRequire/grunt is giving you so I can better tackle it ?

[englercj]

Windows 8, running everything under git-bash. Here is my node env:

$ node --version
v0.8.12

$ npm --version
1.1.63

$ npm ls | grep grunt
├─┬ grunt@0.4.1

$ npm ls | grep urequire
├─┬ grunt-urequire@0.6.0
│ └─┬ urequire@0.6.4

My default grunt command is defined as:

grunt.registerTask('default', ['jshint', 'build']);
grunt.registerTask('build', ['urequire:dev', 'urequire:dist', 'replace:dist']);

Now for some reason I am unable to reproduce it stopping after the first! Maybe I updated and it fixed the issue? If it happens again I will reopen the issue and give you more feedback.

[anodynos]

Glad its OK, hope it doesn't come up again. uRequire v0.5.x had some issues with multiple bundles - perhaps it was that.

Do let me know how the watch-ing goes - its a no brainer to setup and it works like charm!

[englercj]

Yeah that shouldn't be too big of an issue :) Thanks again!

[englercj]

FYI: I am still getting the issue where my grunt tasks stop after a urequire task. Everytime it happens I change my settings to debugLevel: 100, verbose: true so I can capture output for you; and 100% of the time that fixes it.

I can run my tasks repeatedly having it fail, but as soon as I turn up debugging it works fine. The weird part is it seems to complete the task just fine, the bundle is created; it just stops doing anything else.

$ npm ls | grep urequire
├─┬ grunt-urequire@0.6.0
│ └─┬ urequire@0.6.5

$ grunt build:dist
Running "build:dist" (build) task

Running "urequire:dist" (urequire) task
  [urequire/process/Bundle] : Combined file 'build/gf.min.js' written successfully for build #1, rjs.optimize took 7.629secs .

The build:dist should run ['urequire:dist', 'concat:dist, 'replace:dist'] but it stops after the urequire task and leaves the gf.min.js___temp folder still there (but empty). Maybe this is an issue on windows where it silently fails about not being able to remove the folder or something?

It is killing me to keep having to turn on debug logging to get it to run, and I don't want to have that on all the time.

[anodynos]

Certainly this shouldn't happen

Maybe this is an issue on windows where it silently fails about not being able to remove the folder

I'm using wrench.rmdirSyncRecursive (but not tested on windows lately:-). Perhaps its that, I will check it.

Please try:

• running with debugLevel: 40 (the debugLevelSkipTempDeletion threshold is 50, and debug message is 40. Check if it halts right at message 'Deleting temporary directory...'
• Try it on watch, withdebugLevel: 0 - on watch mode it does NOT delete the temp dir, debuggin' or not.

We'll hammer it!

Cheers

BTW: you dont need 'grunt-replace' just for VERSION, check https://github.com/anodynos/uBerscore/blob/46cc6541055cf5d94b60f19f367eedb4b5b20deb/Gruntfile.coffee#L95-L99

[englercj]

Next time it happens I will make sure to turn up to 40 and give you a log.

Note that it never happens to me when running watch tasks.

Thanks for the tip about version, removed that dependency!

[englercj]

@anodynos Got it to happen again! Here are the final lines when run at debug level 40:

  [urequire/process/Bundle] : Combined file 'build/gf.js' written successfully for build #1, rjs.optimize took 2.661secs .
  [urequire/process/Bundle] DEBUG: (40) Deleting temporary directory 'build/gf.js___temp'.

and here is a full log:

https://gist.github.com/englercj/54e7932bff7452a13ff9

Looks like it is a silent error deleting the directory!

[englercj]

This is definately an error with removing the parent directory. Inside wrench.rmdirSyncRecursive the final line of the function is:

    /*  Now that we know everything in the sub-tree has been deleted, we can delete the main
     directory. Huzzah for the shopkeep. */
    return fs.rmdirSync(path);

If I change that to be a try catch:

    try {
        fs.rmdirSync(path);
    } catch(err) {
        console.log('ERROR:', err);
        throw err;
    }

I log an ENOTEMPTY error. Not sure how the error from fs.rmdirSync is being eaten, maybe something in grunt/urequire but the final call that deletes the original path you pass is throwing an ENOTEMPTY error for sure. Still investigating how to fix...

Edit

Looks like others have had this issue on windows using wrench: https://github.com/ryanmcgrath/wrench-js/issues/34

One user decided to use rimraf (https://github.com/isaacs/rimraf) instead, which seems to work fine under windows.

Possible solution: replace wrench with rimraf and mkdirp. Both modules are specific and battle-tested.