webpack-contrib / cache-loader

[DEPRECATED] Caches the result of following loaders on disk
MIT License
638 stars 51 forks source link

perf(index): use the `compiler`'s cached `fs` for stats (`this.fs.stat`) #42

Closed gusvargas closed 6 years ago

gusvargas commented 6 years ago

Related to https://github.com/webpack-contrib/cache-loader/issues/35

I was experiencing a similar issue to that described in https://github.com/webpack-contrib/cache-loader/issues/10#issuecomment-374986963

In my experience, with cache-loader rebuild time increased dramatically – from 2.5s to 9.5s. While initial build time decreased for ~40s.

After profiling incremental builds it looks like fs.stat is the biggest bottle neck, specifically when validating a cache entry after a hit:

https://github.com/webpack-contrib/cache-loader/blob/74fccc4407750942c8a082906309f8af298c16e8/src/index.js#L112-L114

This change decreased my incremental build times back down below 1s.

jsf-clabot commented 6 years ago

CLA assistant check
All committers have signed the CLA.

codecov[bot] commented 6 years ago

Codecov Report

Merging #42 into master will not change coverage. The diff coverage is 0%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #42   +/-   ##
=======================================
  Coverage   17.47%   17.47%           
=======================================
  Files           2        2           
  Lines         103      103           
  Branches       13       13           
=======================================
  Hits           18       18           
  Misses         73       73           
  Partials       12       12
Impacted Files Coverage Δ
src/index.js 17.64% <0%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 74fccc4...4a7ff10. Read the comment docs.

gusvargas commented 6 years ago

If anyone is interested in trying this change out I created a fork of an open source app here: https://github.com/gusvargas/hackernews-react-graphql

Steps to reproduce:

  1. yarn && yarn dev
  2. Navigate to localhost:3000 (I think there is some magic in nextjs that makes it so that only "active" modules are compiled)
  3. Make a trivial change to "Main.js" and observe the terminal output. For me, compile time was floating slightly above and below 100ms.
  4. Uncomment the webpack config change in next.config.js to enable the cache-loader
  5. Restart the dev server and repeat step 3. Now I see compile times around 600ms-700ms.

Linking against the changes in PR brings the compile times back down to what is observed in step 3.

gusvargas commented 6 years ago

@michael-ciniawsky Any chance we can get the 1.2.3 release soon? The incremental build performance issue was the only thing holding us back from using the cache-loader – we'd love to start taking advantage of it ASAP.

michael-ciniawsky commented 6 years ago

Sry for the delay, released in v1.2.3 🎉