assetgraph / assetgraph-builder

AssetGraph-based build system for web apps and web pages.
BSD 3-Clause "New" or "Revised" License
489 stars 42 forks source link

buildProduction is doing too much and is slow #144

Open Munter opened 10 years ago

Munter commented 10 years ago

I have been exposing buildProduction to some more use cases. One of them is https://github.com/jsconf/2014.jsconf.eu

I've cloned the repos and built the page with jekyll. Then ran buildProduction like so:

munter at pollex in ~/git/2014.jsconf.eu on gh-pages
$ buildProduction _site/*.html --outroot build
Guessing --root from input files: file:///home/munter/git/2014.jsconf.eu/_site
 ✔ 0.002 secs: logEvents
 ✔ 0.001 secs: registerRequireJsConfig
 ✔ 0.005 secs: registerLabelsAsCustomProtocols
 ✔ 1.277 secs: loadAssets
 ✔ 0.154 secs: populate
 ✔ 0.011 secs: assumeRequireJsConfigHasBeenFound
 ⚠ WARN: _site/sponsors/images/bloomberg.svg: Parse error in _site/sponsors/images/bloomberg.svg
         entity not found:&ns_svg;
         @#[line:undefined,col:undefined]
 ✔ 187.192 secs: populate
 ✔ 0.045 secs: fixBaseAssetsOfUnresolvedOutgoingRelationsFromHtmlFragments
 ✔ 0.007 secs: assumeThatAllHtmlFragmentAssetsWithoutIncomingRelationsAreNotTemplates
 ✔ 27.597 secs: populate
 ✔ 0.034 secs: removeRelations
 ✔ 0.065 secs: addDataVersionAttributeToHtmlElement
 ✔ 0.018 secs: replaceDartWithJavaScript
 ✔ 0.012 secs: populate
 ✔ 0.035 secs: compileLessToCss
 ✔ 0.039 secs: removeRelations
 ✔ 28.996 secs: populate
 ✔ 29.004 secs: compileJsxToJs
 ✔ 0.098 secs: stripDebug
 ✔ 0.020 secs: removeRelations
 ✔ 2.270 secs: externalizeRelations
 ✔ 0.006 secs: mergeIdenticalAssets
 ✔ 0.131 secs: processImages
 ✔ 0.038 secs: spriteBackgroundImages
 ✔ 0.115 secs: processImages
 ℹ INFO: minifySvgAssetsWithSvgo: Found 2 svg asset(s), but no svgo module is available. Please use npm to install svgo in your project so minifySvgAssetsWithSvgo can require it.
 ✔ 0.041 secs: minifySvgAssetsWithSvgo
 ✔ 63.806 secs: inlineKnockoutJsTemplates
 ✔ 0.407 secs: liftUpJavaScriptRequireJsCommonJsCompatibilityRequire
 ✔ 12.199 secs: flattenRequireJs
 ✔ 3.135 secs: removeUnreferencedAssets
 ✔ 4.384 secs: convertCssImportsToHtmlStyles
 ✔ 0.106 secs: removeDuplicateHtmlStyles
 ✔ 6.176 secs: mergeIdenticalAssets
 ✔ 30.558 secs: bundleRelations
 ✔ 0.040 secs: splitCssIfIeLimitIsReached
 ✔ 0.057 secs: replaceRequireJsWithAlmond
 ✔ 24.387 secs: bundleRelations
 ✔ 12.394 secs: mergeIdenticalAssets
 ✔ 2.216 secs: ngmin
 ✔ 0.020 secs: removeNobundleAttribute
 ✔ 9.192 secs: inlineCssImagesWithLegacyFallback
 ✔ 0.018 secs: mergeIdenticalAssets
 ✔ 121.220 secs: minifyAssets
 ✔ 0.006 secs: replaceSymbolsInJavaScript
 ✔ 0.364 secs: compressJavaScript
 ✔ 0.005 secs: removeEmptyJavaScripts
 ✔ 0.023 secs: removeEmptyStylesheets
 ✔ 15.349 secs: inlineRelations
 ✔ 75.147 secs: inlineAngularJsTemplates
 ✔ 2.068 secs: duplicateFavicon
 ✔ 0.000 secs: setAsyncOrDeferOnHtmlScripts
 ✔ 0.028 secs: omitFunctionCall
 ✔ 0.033 secs: inlineRelations
 ✔ 5.033 secs: moveAssetsInOrder
 ✔ 636.133 secs: buildProduction
 ✔ 2.855 secs: writeAssetsToDisc
  Html 153  6.6 MB
   Css   1 11.1 KB
   Ico   2 13.4 KB
   Png  36  4.9 MB
  Jpeg  70 18.4 MB
 Asset   4 41.0 KB
   Svg   2 38.6 KB
Total: 268 30.1 MB
 ✔ 0.022 secs: writeStatsToStderr

Note all the time spent in transforms that should have no impact at all. The does not use neither Knockout nor Angular nor React, yet a lot of time is spent in those transforms. Why?

Munawwar commented 9 years ago

I've been looking at this issue. I've looked at inlineKnockoutJsTemplates transform, and found that each call to assetGraph.eachAssetPreOrder() method takes anywhere between 150 ms to 700 ms to complete (which includes the calls to assetGraph.findRelations). A assetGraph.findAssets call takes ~ 5ms. A assetGraph.findRelations call takes between 5ms to 15ms. So the bottleneck in that transform is eachAssetPreOrder(). I'll try to dig in further soon. EDIT: Correction. hmm..the slowness maybe because of the O(n^3) nature of the transform.