GoogleChrome / workbox

📦 Workbox: JavaScript libraries for Progressive Web Apps
https://developers.google.com/web/tools/workbox/
MIT License
12.27k stars 804 forks source link

Integration with the webpack DllPlugin #1348

Closed jeffposnick closed 4 months ago

jeffposnick commented 6 years ago

Library Affected: workbox-webpack-plugin

@guymestef wrote at https://github.com/GoogleChrome/workbox/issues/1341#issuecomment-370746634:

I used ManifestTransform and glob* because I'm generated a vendors bundle with the DllPlugin of webpack in a separate build, in order to speed up the build of my app during development. Is it planned or already done for workbox v3 to deal with the use of DllPlugin/DllReferencePlugin?

@guymestef, could you share the configuration you're using for DllPlugin—I gather it's in a separate webpack build?

The docs for DllPlugin are at https://webpack.js.org/plugins/dll-plugin/, and from reading that, it looks like it spits out its own manifest.json file that we might be able to repurpose as a source of files to precache.

Guymestef commented 6 years ago

You are right. I'm using two webpack config files:

  1. webpack.config.vendor.js
    
    const path = require('path');
    const webpack = require('webpack');
    const ExtractTextPlugin = require('extract-text-webpack-plugin');
    const BundleAnalyzerPlugin = require('webpack-bundle-analyzer').BundleAnalyzerPlugin;

module.exports = (env) => { const analyzeBundle = env && env.analyzeBundle; const extractCss = new ExtractTextPlugin('vendor.css'); return [{ stats: { modules: false }, resolve: { extensions: ['.js'] }, module: { rules: [{ test: /.(png|jpg|jpeg|gif|woff|woff2|eot|ttf|svg)(\?|$)/, use: 'url-loader?limit=100000' }, { test: /.scss$/, use: ExtractTextPlugin.extract({ fallback: 'style-loader', use: [{ loader: 'css-loader', options: { importLoaders: 1 } }, { loader: 'sass-loader' }] }) }, { test: /.less$/, use: ExtractTextPlugin.extract({ fallback: 'style-loader', use: [{ loader: 'css-loader', options: { importLoaders: 1 } }, { loader: 'less-loader' }] }) }, { test: /.css$/, use: ExtractTextPlugin.extract({ fallback: 'style-loader', use: 'css-loader' }) }, { test: require.resolve('jquery'), use: [{ loader: 'expose-loader', query: 'jQuery' }, { loader: 'expose-loader', query: '$' }] }, { test: require.resolve('moment'), use: [{ loader: 'expose-loader', query: 'moment' }] }, { test: require.resolve('knockout'), use: [{ loader: 'expose-loader', query: 'ko' }] } ] }, entry: { vendor: [ 'popper.js', 'bootstrap', 'bootstrap/scss/bootstrap.scss', 'bootstrap-social', 'knockout', 'knockout-mapping', 'knockout-postbox', 'knockout.validation', 'event-source-polyfill', 'isomorphic-fetch', 'jquery',

            'babel-polyfill',
            'stacktrace-js',

            'i18next',

            'jquery-validation',
            'jquery-validation/dist/additional-methods',
            'jquery-validation-unobtrusive',
            'css-toggle-switch/dist/toggle-switch.css',
            'moment',
            'tempusdominus-bootstrap-4',
            'tempusdominus-bootstrap-4/build/css/tempusdominus-bootstrap-4.css',
            'bootstrap-tagsinput',
            'bootstrap-tagsinput/dist/bootstrap-tagsinput.less',
            'font-awesome/less/font-awesome.less',
            'd3',
            'nvd3',
            'nvd3/build/nv.d3.css',
            'jquery-ui',
            'jquery-ui/ui/widgets/draggable',
            'jquery-ui-touch-punch',
            'iscroll/build/iscroll-zoom',
            'jquery.cookie',

            'offline-js',
            'offline-js/themes/offline-theme-chrome.css',

            'file-size',

            'jquery.complexify/jquery.complexify.banlist',
            'jquery.complexify',
            'jquery.payment',
            'mangopay-cardregistration-js-kit',
            'select2',
            'select2/dist/css/select2.css',
            'select2-bootstrap-theme/dist/select2-bootstrap.css',
            'openlayers',
            'openlayers/dist/ol.css',
            'google-maps',
            'clipboard',
            'lightbox2',
            'lightbox2/dist/css/lightbox.css',
            'aos',
            'aos/dist/aos.css',
            'animate.css/animate.css',
            'screenfull',
            'dotdotdot',
            'youtube-player',
            'jquery-nearest'
        ]
    },
    output: {
        path: path.join(__dirname, 'wwwroot', 'dist'),
        publicPath: '/dist/',
        filename: '[name].js',
        library: '[name]_[hash]'
    },
    plugins: [
        extractCss,
        new webpack.ProvidePlugin({
            $: 'jquery',
            jQuery: 'jquery',
            moment: 'moment'
        }), // Maps these identifiers to the jQuery package (because Bootstrap expects it to be a global variable)
        new webpack.DllPlugin({
            path: path.join(__dirname, 'wwwroot', 'dist', '[name]-manifest.json'),
            name: '[name]_[hash]'
        })
    ]
}];

};

2. webpack.config.js
```javascript
const path = require('path');
const webpack = require('webpack');
const CleanWebpackPlugin = require('clean-webpack-plugin');
const ExtractTextPlugin = require('extract-text-webpack-plugin');
const WebpackPwaManifest = require('webpack-pwa-manifest');
const BundleAnalyzerPlugin = require('webpack-bundle-analyzer').BundleAnalyzerPlugin;
const {
    GenerateSW
} = require('workbox-webpack-plugin');
const ForkTsCheckerWebpackPlugin = require('fork-ts-checker-webpack-plugin');
const bundleOutputDir = './wwwroot/dist';

module.exports = (env) => {
    const isDevBuild = !(env && env.prod);
    return [{
        stats: {
            modules: false
        },
        entry: {
            'main': ['./ClientApp/boot.js'].concat(isDevBuild ? ['./ClientApp/offline-simulate-ui.min.js'] : []),
            'vendor.config.fr': './ClientApp/vendor.config.fr.js',
            'vendor.config.es': './ClientApp/vendor.config.es.js',
            'vendor.config.de': './ClientApp/vendor.config.de.js',
            'vendor.config.en': './ClientApp/vendor.config.en.js',
            'vendor.config.it': './ClientApp/vendor.config.it.js',
            'vendor.config.pt': './ClientApp/vendor.config.pt.js'
        },
        resolve: {
            extensions: ['.ts', '.js', '.json']
        },
        output: {
            path: path.join(__dirname, bundleOutputDir),
            filename: '[name].js',
            publicPath: '/dist/'
        },
        module: {
            rules: [{
                    test: /locales/,
                    loader: '@alienfast/i18next-loader',
                    query: {
                        basenameAsNamespace: true
                    }
                },
                {
                    test: /\.ts$/,
                    include: /ClientApp/,
                    use: [{
                        loader: 'ts-loader',
                        options: {
                            transpileOnly: true
                        }
                    }]
                },
                {
                    test: /\.html$/,
                    use: 'raw-loader'
                },
                {
                    test: /\.scss$/,
                    use: ExtractTextPlugin.extract({
                        fallback: 'style-loader',
                        use: [{
                            loader: 'css-loader',
                            options: {
                                importLoaders: 1
                            }
                        }, {
                            loader: 'sass-loader'
                        }]
                    })
                },
                {
                    test: /\.css$/,
                    use: ExtractTextPlugin.extract({
                        fallback: 'style-loader',
                        use: 'css-loader'
                    })
                },
                {
                    test: /\.(png|jpg|jpeg|gif|svg)$/,
                    use: 'url-loader?limit=25000'
                },
                {
                    test: /\.js$/,
                    exclude: /(node_modules|bower_components)/,
                    use: {
                        loader: 'babel-loader',
                        options: {
                            presets: ['@babel/preset-env']
                        }
                    }
                },
                {
                    test: /\.worker.js$/,
                    use: [{
                        loader: 'worker-loader',
                        options: {
                            inline: true
                        }
                    }]
                },
                {
                    test: /\.html$/,
                    use: [{
                        loader: 'html-loader'
                    }]
                }
            ]
        },
        plugins: [
                new CleanWebpackPlugin([
                    'wwwroot/dist/precache-manifest.*.js',
                    'wwwroot/dist/manifest.json',
                    'wwwroot/dist/icons',
                    'wwwroot/dist/workbox*'
                ]),
                new ForkTsCheckerWebpackPlugin(),
                new webpack.DllReferencePlugin({
                    context: __dirname,
                    manifest: require('./wwwroot/dist/vendor-manifest.json')
                }),
                new ExtractTextPlugin({
                    filename: '[name].css'
                }),
                new WebpackPwaManifest({
                    filename: 'manifest.json',
                    name: 'MesHĂ´tes.com ',
                    short_name: 'MesHĂ´tes',
                    description: 'A wedding seating chart? A table plan for a birdthday or a gala? Make your seating plan online and let our robot Nono place your guests for you',
                    background_color: '#e71d79',
                    fingerprints: false,
                    icons: [{
                            src: path.resolve('ClientApp/css/images/icon.png'),
                            sizes: [120, 152, 167, 180, 1024],
                            destination: path.join('icons', 'ios'),
                            ios: true
                        },
                        {
                            src: path.resolve('ClientApp/css/images/icon.png'),
                            size: 1024,
                            destination: path.join('icons', 'ios'),
                            ios: 'startup'
                        },
                        {
                            src: path.resolve('ClientApp/css/images/icon.png'),
                            sizes: [36, 48, 72, 96, 144, 192, 512],
                            destination: path.join('icons', 'android')
                        }
                    ]
                })
            ].concat(isDevBuild ? [
                // Plugins that apply in development builds only
                new webpack.SourceMapDevToolPlugin({
                    filename: '[file].map', // Remove this line if you prefer inline source maps
                    moduleFilenameTemplate: path.relative(bundleOutputDir,
                        '[resourcePath]') // Point sourcemap entries to the original file locations on disk
                })
            ] : [])
            .concat([new GenerateSW({
                swDest: 'service-worker.js',
                manifestTransforms: [
                    (manifestEntries) => ({
                        manifest: manifestEntries.map((entry) => {
                            if (entry.url.startsWith('dist/')) {
                                entry.url = entry.url.replace(/^dist\//i, '');
                            }
                            return entry;
                        })
                    })
                ],
                cacheId: 'sp-admin',
                importWorkboxFrom: 'local',
                maximumFileSizeToCacheInBytes: 10 * 1024 * 1024,
                globDirectory: './wwwroot',
                globPatterns: ['**/*.{js,png,jpg,gif,ico,html,css,eot,svg,ttf,json}'],
                globIgnores: [
                    '**/vendor-manifest.json',
                    '**/workbox*.js',
                    '**/*.hot-update.js',
                    '**/*.hot-update.json',
                    '**/precache-manifest.*.js'
                ],
                clientsClaim: true,
                skipWaiting: true
            })])
    }];
};

When I'm adding a vendor that I know will be fully embedded in my app, I add it to the list in webpack.config.vendor.js. And I run webpack to generate vendor.js, vendor.css and copy to dist folder all required files (images, font, ...). Webpack DllPlugin generate a vendor-manifest.json file that will be used by DllReferencePlugin in the webpack.config.js build.

djeeg commented 6 years ago

in order to speed up the build of my app during development

The DllPlugin is really only used for development right? In production either CommonChunksPlugin (webpack 3) or splitChunks (webpack 4) are used for more optimised code splitting

What is the benefit of combining DllPlugin and Workbox in development?

Instead, maybe create a third webpack environment, an extra DEV configuration, so service worker testing is verifying the actual chunks you are expecting in production. It may be a little slower to compile without DllPlugin, but at least you are testing closer to the chunks you expect in production

eg

Guymestef commented 6 years ago

Since my previous post, I've implemented splitChunks too. DllPlugin allows to pre-built dependencies in a separate bundle and optimizations are applied on the generated bundle as well, like for a classic webpack build (except for tree shaking of course).

So basically, I've:

I agree that in PROD, DllPlugin can be just removed to only use splitChunks (probably better optimizations with tree shaking, ...)

But I prefer to keep workbox in DEV to have a DEV env as close as possible as the PROD env. For workbox, it allows to see the impacts/side effects of runtime caching during the development of a feature, instead of discovering them after the development while testing the app. (I'm experimenting caching on particular GET requests on my app) And I don't think having multiple DEV configurations is a good idea, since I'm sure that people will always use the same one.

djeeg commented 6 years ago

Oh you are using DllPlugin in PROD, makes sense now for the service workers, please ignore my comment.

I recommend to you thought, being able to swap between multiple DEV configs. Optimal dev tool operation generally conflicts with configuration required by other dev tools. Compile speed vs runtime speed vs bundle size vs source map vs HMR vs service worker ... etc

jeffposnick commented 4 years ago

This issue's been open for a few years now, and my hope is that it's addressed by the workbox-webpack-plugin v5. Please give it a try with v5, and if it's still an issue, we can reopen this bug.

steven87vt commented 3 years ago

so i have this problem still. im using "workbox-webpack-plugin": "^5.1.4"

basically we have a vendor build and an app build:

# from vendor
                                    Asset      Size  Chunks                                Chunk Names
     205f07b3883c484f27f40d21a92950d4.ttf   200 KiB          [emitted]                     
     2f12242375edd68e9013ecfb59c672e9.svg   730 KiB          [emitted]              [big]  
     3602b7e8b2cb1462b0bef9738757ef8a.svg   141 KiB          [emitted]                     
    4451e1d86df7491dd874f2c41eee1053.woff   102 KiB          [emitted]                     
     664de3932dd6291b4b8a8c0ddbcb4c61.svg   896 KiB          [emitted]              [big]  
     8300bd7f30e0a313c1d772b49d96cb8e.ttf   133 KiB          [emitted]                     
     8ac3167427b1d5d2967646bd8f7a0587.eot   200 KiB          [emitted]                     
     e2ca6541bff3a3e9f4799ee327b28c58.eot   134 KiB          [emitted]                     
           vendor.7b437f303d916fdefeca.js   293 KiB       0  [emitted] [immutable]  [big]  vendor
                               vendor.css  2.73 MiB       0  [emitted]              [big]  vendor

# from build
                                 Asset      Size  Chunks                                Chunk Names
                               my.docx    15 KiB          [emitted]                     
                             my_logo.png  11.4 KiB          [emitted]                     
                              favicon.ico  1.12 KiB          [emitted]                     
                          header_logo.png    27 KiB          [emitted]                     
                               index.html   1.3 KiB          [emitted]                     
                        main~19a26b3e.css  49.2 KiB       0  [emitted]                     main~19a26b3e
    main~19a26b3e.f2ab6cc05c381c176983.js  7.71 MiB       0  [emitted] [immutable]  [big]  main~19a26b3e
    main~203e0718.62b9033a01ff8309f310.js    13 MiB       1  [emitted] [immutable]  [big]  main~203e0718
    main~678f84af.485197aa3ff38c5d892a.js  13.4 MiB       2  [emitted] [immutable]  [big]  main~678f84af
    main~748942c6.446315a7970b3be9bcf8.js  6.62 MiB       3  [emitted] [immutable]  [big]  main~748942c6
    main~b5906859.e5e8694f638ec4597190.js   766 KiB       4  [emitted] [immutable]  [big]  main~b5906859
    main~fdc6512a.2ecc0d9a19eabd4bc993.js   5.8 MiB       5  [emitted] [immutable]  [big]  main~fdc6512a
                                    sw.js   710 KiB          [emitted]              [big]

everything is configured correctly i assume, because all of our vendor packages imports work like vuex, bootstrap, etc.

plugins: [
      new VueLoaderPlugin(),
      new CopyWebpackPlugin([
        {
          from: './ClientApp/images'
        }
      ]),
      new webpack.DllReferencePlugin({
        context: __dirname,
        manifest: require('./wwwroot/dist/vendor-manifest.json')
      }),
      new HtmlWebpackPlugin({
        title: 'MYTITLE',
        template: 'ClientApp/templates/index.template.ejs'
      }),
      new InjectManifest({
        swSrc: './ClientApp/sw/sw.js',
        swDest: 'sw.js',
        maximumFileSizeToCacheInBytes: maxChunkCacheSize
      })
    ]

this is the manifest injected into the service worker:

[
  {
    "revision": "edd825aefbed96fed840f30d2fc8a439",
    "url": "/my.docx"
  },
  {
    "revision": "128a9a1e9d08815847191ae099565c72",
    "url": "/my_logo.png"
  },
  {
    "revision": "c1ba78fd0984aa4f779cddcb042f9670",
    "url": "/favicon.ico"
  },
  {
    "revision": "7f5437f1584f0dd1ca8318224690626b",
    "url": "/header_logo.png"
  },
  {
    "revision": "4211d09469c018345b8a4fc6ea573dff",
    "url": "/index.html"
  },
  {
    "revision": "87aa2377e2defbb85a61c193eedab976",
    "url": "/main~._e.js"
  },
  {
    "revision": "8c1335d24eee765456edf3042e79c511",
    "url": "/main~._node_modules_d.js"
  },
  {
    "revision": "7d4d3bb0ef5a8c0136c2c819b5e726af",
    "url": "/main~._node_modules_v.js"
  }
]

i original attempted to use config.additionalManifestEntries but saw i needed to generate the revision information and quickly moved on. i then took a look at the config.include option thinking that it would reference additional files as well as the build assets, only to debug the code and come to the conclusion that its only for the ascertained assets from the current webpack build.

const isIncluded = !Array.isArray(config.include) || checkConditions(asset, compilation, config.include);
if (!isIncluded) {
  continue;
} // If we've gotten this far, then add the asset.
filteredAssets.add(asset);

so i borrowed some of your logic for ascertaining the asset and its hash

const bundleOutputDir = './wwwroot/dist'

const path = require('path')
const glob = require('glob')
const webpack = require('webpack')
const fs = require('fs')
const { InjectManifest } = require('workbox-webpack-plugin')
const { RawSource } = require('webpack-sources')
const getAssetHash = require('workbox-webpack-plugin/build/lib/get-asset-hash')

...
module.exports = async (env) => {

  const findVendorFiles = function (root) {
    const anyFiles = [
      // globed files from paths
      // glob does not yet support multiple extensions
      // https://github.com/isaacs/node-glob/issues/217
      { key: 'script', glob: glob.sync(`${root}/vendor.*.js`) },
      { key: 'svg', glob: glob.sync(`${root}/*.svg`) },
      { key: 'ttf', glob: glob.sync(`${root}/*.ttf`) },
      { key: 'woff', glob: glob.sync(`${root}/*.woff`) },
      { key: 'eot', glob: glob.sync(`${root}/*.eot`) }
    ]
    const files = [
      // actual files by name
      { name: 'vendor-manifest.json', path: path.resolve(`${root}/vendor-manifest.json`), url: '/vendor-manifest.json', revision: undefined },
      { name: 'vendor.css', path: path.resolve(`${root}/vendor.css`), url: '/vendor.css', revision: undefined }
    ]

    const getFileName = function (file) {
      const paths = file.split('/')
      return paths[paths.length - 1]
    }

    const onError = function (err) {
      console.error(err)
    }

    const vendorJsGlob = anyFiles[0].glob
    if (!vendorJsGlob || vendorJsGlob.length !== 1) {
      console.error('#### ANNOYING BUILD ERROR - NOT CONFIGURED FOR MULTIPLE OR NULL VENDOR.[HASH].JS FILES.', vendorJsGlob)
      console.error('#### IF YOU ARE ATTEMPTING TO CHUNK THE VENDOR PACKAGES YOU NEED TO ITERATE THE OPTIONS IN INDEX.HTML.EJS AS IT ASSUMES 1 FILE')
      throw new Error('you shall not pass!')
    }

    const promises = []

    files.forEach((file) => {
      promises.push(
        new Promise((resolve, reject) => {
          fs.readFile(file.url, (err, source) => {
            if (!err) {
              file.revision = getAssetHash(new RawSource(source))
              resolve()
              return
            }

            onError(err)
            reject(err)
          })
        })
      )
    })

    // for every data point in globs
    anyFiles.forEach((data) => {
      let { glob } = data
      // for every file found by glob
      glob.forEach((file) => {
        let name = getFileName(file)
        let url = `/${name}`
        promises.push(
          new Promise((resolve, reject) => {
            fs.readFile(file, (err, source) => {
              if (!err) {
                files.push({ name, url, revision: getAssetHash(new RawSource(source)) })
                resolve()
                return
              }

              onError(err)
              reject(err)
            })
          })
        )
      })
    })

    return Promise.all(promises).then(() => files)
  }

  const outputPath = path.join(__dirname, bundleOutputDir)
  const allVendorFiles = await findVendorFiles(outputPath)
  const vendorJsFileName = allVendorFiles.find((file) => {
    return /vendor\..*\.js/.test(file.name)
  }).name

  const additionalManifestEntries = allVendorFiles.map(({ url, revision }) => {
    return {
      url,
      revision
    }
  })

  const options = [{
    plugins: [
        new VueLoaderPlugin(),
        new CopyWebpackPlugin([{ from: './ClientApp/images', to: outputPath }], {
          concurrency: 100,
          logLevel: debug ? 'debug' : 'warn',
          copyUnmodified: true
        }),
        new webpack.DllReferencePlugin({
          context: __dirname,
          manifest: './wwwroot/dist/vendor-manifest.json'
        }),
        new HtmlWebpackPlugin({
          title: 'MYTITLE',
          customOptions: {
            vendorJs: vendorJsFileName
          },
          template: 'ClientApp/templates/index.template.ejs'
        }),
        new InjectManifest({
          swSrc: './ClientApp/sw/sw.js',
          swDest: 'sw.js',
          maximumFileSizeToCacheInBytes: maxChunkCacheSize,
          additionalManifestEntries
        })
      ]
}]
 ...

and behold

[
  {
    "revision": "205f07b3883c484f27f40d21a92950d4",
    "url": "/205f07b3883c484f27f40d21a92950d4.ttf"
  },
  {
    "revision": "2f12242375edd68e9013ecfb59c672e9",
    "url": "/2f12242375edd68e9013ecfb59c672e9.svg"
  },
  {
    "revision": "3602b7e8b2cb1462b0bef9738757ef8a",
    "url": "/3602b7e8b2cb1462b0bef9738757ef8a.svg"
  },
  {
    "revision": "4451e1d86df7491dd874f2c41eee1053",
    "url": "/4451e1d86df7491dd874f2c41eee1053.woff"
  },
  {
    "revision": "664de3932dd6291b4b8a8c0ddbcb4c61",
    "url": "/664de3932dd6291b4b8a8c0ddbcb4c61.svg"
  },
  {
    "revision": "8300bd7f30e0a313c1d772b49d96cb8e",
    "url": "/8300bd7f30e0a313c1d772b49d96cb8e.ttf"
  },
  {
    "revision": "8ac3167427b1d5d2967646bd8f7a0587",
    "url": "/8ac3167427b1d5d2967646bd8f7a0587.eot"
  },
  {
    "revision": "edd825aefbed96fed840f30d2fc8a439",
    "url": "/my.docx"
  },
  {
    "revision": "128a9a1e9d08815847191ae099565c72",
    "url": "/my_logo.png"
  },
  {
    "revision": "e2ca6541bff3a3e9f4799ee327b28c58",
    "url": "/e2ca6541bff3a3e9f4799ee327b28c58.eot"
  },
  {
    "revision": "c1ba78fd0984aa4f779cddcb042f9670",
    "url": "/favicon.ico"
  },
  {
    "revision": "7f5437f1584f0dd1ca8318224690626b",
    "url": "/header_logo.png"
  },
  {
    "revision": "95198d80e58c2668caf9355f102091bc",
    "url": "/index.html"
  },
  {
    "revision": "87aa2377e2defbb85a61c193eedab976",
    "url": "/main~._e.2b5c0027f1020ffd25eb.js"
  },
  {
    "revision": "8c1335d24eee765456edf3042e79c511",
    "url": "/main~._node_modules_d.ab5802ca71b75ab23c9d.js"
  },
  {
    "revision": "7d4d3bb0ef5a8c0136c2c819b5e726af",
    "url": "/main~._node_modules_v.1578962663fb55e68199.js"
  },
  {
    "revision": "7b08f859f3e7280a710eab795b31a443",
    "url": "/vendor-manifest.json"
  },
  {
    "revision": "4ba04dff11ff8916dd57b7c246aafba1",
    "url": "/vendor.css"
  },
  {
    "revision": "21eb37fdc7457781e350efaa026cc367",
    "url": "/vendor.f8532336d4a3331b4282.js"
  }
]

which i have to say is a pain in the 🤬. am i missing something like a pre InjectManifest plugin that spits out partial manifest files from previous builds that are consumed as part of the current build? If not would that be a logical way to solve this?

steven87vt commented 3 years ago

So another update. Looking back at our git history has revealed back when we used version 4 we depended on the following InjectManifest plugin options:

https://developers.google.com/web/tools/workbox/reference-docs/latest/module-workbox-build#.injectManifest

Parameter Type Description
globDirectory string The local directory you wish to match globPatterns against. The path is relative to the current directory.
globPatterns Array Files matching any of these patterns will be included in the precache manifest. For more information, see the glob primer.

When I tried to bring these values back I run into the following errors:

ERROR in Please check your InjectManifest plugin configuration:
    "globDirectory" is not a supported parameter.. "globPatterns" is not a supported parameter.

    ERROR in Cannot read property 'push' of undefined

Removing globDirectory and leaving globPatterns:

ERROR in Please check your InjectManifest plugin configuration:
    "globPatterns" is not a supported parameter.

    ERROR in Cannot read property 'push' of undefined

Both of these parameters are still in what I perceive as the latest v5 documentation:

https://developers.google.com/web/tools/workbox/reference-docs/latest/module-workbox-build#.injectManifest

steven87vt commented 3 years ago

After having a coworker point out that the document link above clearly references 6.1.5 🤦 I upgraded and repeated the build and still see the same error output.

// package.json versions
"webpack": "^4.44.2",
"workbox-webpack-plugin": "~6.1.5"

// package-lock.json
"workbox-webpack-plugin": {
  "version": "6.1.5",
  "resolved": "https://registry.npmjs.org/workbox-webpack-plugin/-/workbox-webpack-plugin-6.1.5.tgz",
  "integrity": "sha512-tsgeNAYiFP4STNPDxBVT58eiU8nGUmcv7Lq9FFJkQf5MMu6tPw1OLp+KpszhbCWP+R/nEdu85Gjexs6fY647Kg==",
  "dev": true,
  "requires": {
    "fast-json-stable-stringify": "^2.1.0",
    "pretty-bytes": "^5.4.1",
    "source-map-url": "^0.4.0",
    "upath": "^1.2.0",
    "webpack-sources": "^1.4.3",
    "workbox-build": "^6.1.5"
  },
  "dependencies": {
    "fast-json-stable-stringify": {
      "version": "2.1.0",
      "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.1.0.tgz",
      "integrity": "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw==",
      "dev": true
    },
    "webpack-sources": {
      "version": "1.4.3",
      "resolved": "https://registry.npmjs.org/webpack-sources/-/webpack-sources-1.4.3.tgz",
      "integrity": "sha512-lgTS3Xhv1lCOKo7SA5TjKXMjpSM4sBjNV5+q2bqesbSPs5FjGmU6jjtBSkX9b4qW87vDIsCIlUPOEhbZrMdjeQ==",
      "dev": true,
      "requires": {
        "source-list-map": "^2.0.0",
        "source-map": "~0.6.1"
      }
    }
  }
}

webpack-config.js

new InjectManifest({
  swSrc: './ClientApp/sw/sw.js',
  swDest: 'sw.js',
  maximumFileSizeToCacheInBytes: 20 * 1024 * 1024,
  globDirectory: './ClientApp/wwwroot/dist',
  globPatterns: ['vendor.css', 'vendor.*.js']
  // additionalManifestEntries,
  // dontCacheBustURLsMatching: /(main|vendor).*\.js$/
})

npm build output

Hash: 2d9a0004d0d32492ac96
Version: webpack 4.44.2
Child
    Hash: 2d9a0004d0d32492ac96
    Time: 43542ms
    Built at: 06/16/2021 12:04:48 PM
                       Asset      Size                 Chunks             Chunk Names
                  daily.docx    15 KiB                         [emitted]  
                my_logo.png  11.4 KiB                         [emitted]  
                 favicon.ico  1.12 KiB                         [emitted]  
             header_logo.png    27 KiB                         [emitted]  
                  index.html  1.14 KiB                         [emitted]  
                 main~._C.js  15.3 MiB               main~._C  [emitted]  main~._C
                 main~._e.js  7.34 MiB               main~._e  [emitted]  main~._e
    main~._node_modules_d.js   811 KiB  main~._node_modules_d  [emitted]  main~._node_modules_d
    main~._node_modules_f.js  13.2 MiB  main~._node_modules_f  [emitted]  main~._node_modules_f
    main~._node_modules_r.js  13.9 MiB  main~._node_modules_r  [emitted]  main~._node_modules_r
    main~._node_modules_v.js  6.69 MiB  main~._node_modules_v  [emitted]  main~._node_modules_v

    ERROR in Please check your InjectManifest plugin configuration:
    "globDirectory" is not allowed

    ERROR in Cannot read property 'push' of undefined
jeffposnick commented 3 years ago

Hello @steven87vt—regarding the supported options, they're documented for v6 at https://developers.google.com/web/tools/workbox/reference-docs/latest/module-workbox-webpack-plugin.InjectManifest#InjectManifest

(you were looking at the workbox-build docs, which support a different set of parameters).

The latest version of workbox-webpack-plugin differs from v4 in that it no longer uses glob patterns against the filesystem to try to determine the list of build assets created by a webpack compilation. Instead, workbox-webpack-plugin obtains its list of assets to precache by looking at the actual assets produced by a webpack compilation.

This might lead to some different issues—like what you've identified with the DllPlugin—but at the same time, it fixed a whole class of bugs related to the plugin not picking up assets that weren't actually written to disk.

steven87vt commented 3 years ago

Awesome thanks for that clarification! So then additionalManifestEntries is the correct way forward with capturing that information?

If additionalManifestEntries is the way to go my hash generation technique employed above (borrowing the hash from workbox) no longer works for me as packages/workbox-webpack-plugin/src/lib/get-asset-hash.js changed to support webpack 5 (somewhere along the line source.source() changed to asset.source.source()). I can work around this but I feel like its probably not the correct way to go.

const fs = require('fs')
const getAssetHash = require('workbox-webpack-plugin/build/lib/get-asset-hash')

fs.readFile(path, (err, source) => {
  // getAssetHash (new RawSource(source))`
    getAssetHash ({source: new RawSource(source)})`
})

All this leads me to the question: do I even need webpack's filename hashing for browser cache in a system where sw is always loaded and calling precacheAndRoute since webpack is already providing the Asset.Info?

output: { 
  "filename": "[name].[contenthash].js"
}

My root issue stems from a situation where some users get site updates and sometimes they get cached changes. Our web server (kestrel) will issue 304 responses for the same files in the manifest (it defaults to using etag headers). Does that effect sw.js precacheAndRoute(manifest) handling on page refresh (cache first?)? When I inspect chrome's application tab/cache data the runtime cache only displays registered routes and has no manifest entries, I hope this is correct.

Maybe I should just merged the vendor and main builds into 1 build, remove the DDL plugin & turn off kestrel's etag support for anything related to index.html and hope for the best?

Any insight is appreciated.

tomayac commented 4 months ago

Hi there,

Workbox is moving to a new engineering team within Google. As part of this move, we're declaring a partial bug bankruptcy to allow the new team to start fresh. We realize this isn't optimal, but realistically, this is the only way we see it working. For transparency, here're the criteria we applied:

Thanks, and we hope for your understanding! The Workbox team