v1.0.0

[melck]

Hello,

Now that we have tests. I want to rewrite plugin to es6 syntax and add following changes :

• Support only webpack >4
• Handle splitChunk option of Webpack to add files related of a chunk.
• Add integrity option to handle https://www.w3.org/TR/SRI/

What do you think ? I'll start a pull request to show you changes that i want to made

[owais]

Sounds good but since we decided to only support webpack>4, we have a good opportunity to improve other things and break backward compatibility.

One thing I'd like to do is to default to relative paths and remove the relative path option. We should also remove the path key from stats file and instead add relativePath and absolutePath keys. This will ensure other tools that consume the stats file break in an obvious way.

[owais]

Another thing I'd like to do is to review the output of stats file and try to make it more generic. I think the stats file shouldn't be specific to webpack. It should be generic so any webpack like tool can generate stats files that confirm to the schema. Django (and other non-JS projects) can then rely on that generic stats file schema to load static assets. This should allow multiple ecosystems to have just a single set of libraries to access static assets generated by any build tool.

What should such a schema for the stats file look like?

[owais]

What other features/change would people like to see before we release 1.0?

[owais]

I'll create a 1.0 milestone once we have a good idea about what 1.0 should look like.

[owais]

One more: https://github.com/owais/webpack-bundle-tracker/issues/56

[melck]

Sounds good but since we decided to only support webpack>4, we have a good opportunity to improve other things and break backward compatibility.

One thing I'd like to do is to default to relative paths and remove the relative path option. We should also remove the path key from stats file and instead add relativePath and absolutePath keys. This will ensure other tools that consume the stats file break in an obvious way.

Do we really need absolute path ? I think path should always be relative from stat file and webpack seems to always avoid absolute path.

Regarding a standard stat file, I think we should capture all the assets and specify for js files if they are entry points or loaded on demand.

Regarding SRI, i have made a working exemple on my branch but i saw after that plugin https://github.com/waysact/webpack-subresource-integrity who do a better job. Maybe we could only use output of this plugin to add integrity to stat file if present.

[cscanlin]

+1

https://github.com/owais/webpack-bundle-tracker/issues/52#issuecomment-572046767 It would be great to have a standard way to track bundles for all webpack (and other) projects.

If this (or any) format could be made into a standard, the create-react-app team may be inclined to include it as part of that project, which would be a big win for users of django-webpack-loader. https://github.com/facebook/create-react-app/issues/99#issuecomment-248883000

[jonalxh]

It is very necessary to allow the compressed bundles load, like .gz and .br.

[melck]

@jonalxh I will add tests with compression-webpack-plugin to see how it behave and fix it if needed.

@cscanlin The way we track bundles come from webpack api v4 reference. If you think about a case that will not work, tell me and i will add tests to check.

[melck]

I have tried compression-webpack-plugin to understand how it works. It seems he only add compressed files to stats.compilation.assets and do not modify chunkGroups. With the plugin option all assets are not compressed because of ratio and treshold. To handle this case, we could modify structure of webpack-stats.json. First add assets entry with all assets generated then simplify chunks entry to refer to a asset name.

Exemple :

I added a test with this webpack-config.json and forced compression on all files:

{
  context: __dirname,
  entry: {
    app1: path.resolve(__dirname, 'fixtures', 'app1.js'),
    appWithAssets: path.resolve(__dirname, 'fixtures', 'appWithAssets.js'),
  },
  output: {
    path: OUTPUT_DIR,
    filename: 'js/[name]-[hash].js',
    publicPath: 'http://localhost:3000/',
  },
  module: {
    rules: [{ test: /\.css$/, use: [MiniCssExtractPlugin.loader, 'css-loader'] }],
  },
  optimization: {
    splitChunks: {
      cacheGroups: {
        vendors: {
          name: 'vendors',
          test: /[\\/]node_modules[\\/]/,
          priority: -10,
          chunks: 'initial',
        },
        commons: {
          name: 'commons',
          test: /[\\/]?commons/,
          enforce: true,
          priority: -20,
          chunks: 'all',
          reuseExistingChunk: true,
        },
        default: {
          name: 'shared',
          reuseExistingChunk: true,
        },
      },
    },
  },
  plugins: [
    new MiniCssExtractPlugin({ filename: 'css/[name]-[hash].css' }),
    new CompressionPlugin({
      filename: '[path].gz[query]',
      test: /\.(js|css)$/,
      threshold: 1,
      minRatio: 1, // Compress all files
      deleteOriginalAssets: false,
    }),
    new CompressionPlugin({
      filename: '[path].br[query]',
      algorithm: 'brotliCompress',
      test: /\.(js|css)$/,
      threshold: 1,
      minRatio: 1, // Compress all files
      deleteOriginalAssets: false,
      compressionOptions: {
        params: {
          [zlib.constants.BROTLI_PARAM_QUALITY]: 11
        }
      }
    }),
    new BundleTrackerPlugin({
      path: OUTPUT_DIR,
      relativePath: true,
      includeParents: true,
    }),
  ],
}

The results of this test give (i forced compression on all files) :

/tmp/wbt-tests-bHBIph/
├── css
│   ├── appWithAssets-7da09931011a19e41c04.css
│   ├── appWithAssets-7da09931011a19e41c04.css.br
│   └── appWithAssets-7da09931011a19e41c04.css.gz
├── js
│   ├── 2-7da09931011a19e41c04.js
│   ├── 2-7da09931011a19e41c04.js.br
│   ├── 2-7da09931011a19e41c04.js.gz
│   ├── app1-7da09931011a19e41c04.js
│   ├── app1-7da09931011a19e41c04.js.br
│   ├── app1-7da09931011a19e41c04.js.gz
│   ├── appWithAssets-7da09931011a19e41c04.js
│   ├── appWithAssets-7da09931011a19e41c04.js.br
│   ├── appWithAssets-7da09931011a19e41c04.js.gz
│   ├── commons-7da09931011a19e41c04.js
│   ├── commons-7da09931011a19e41c04.js.br
│   ├── commons-7da09931011a19e41c04.js.gz
│   ├── vendors-7da09931011a19e41c04.js
│   ├── vendors-7da09931011a19e41c04.js.br
│   └── vendors-7da09931011a19e41c04.js.gz
└── webpack-stats.json

I have change webpack-tracker.json structure who give now :

{
  "status": "done",
  "chunks": {
    "app1": [
      "js/vendors-7da09931011a19e41c04.js",
      "js/commons-7da09931011a19e41c04.js",
      "js/app1-7da09931011a19e41c04.js"
    ],
    "appWithAssets": [
      "js/vendors-7da09931011a19e41c04.js",
      "js/commons-7da09931011a19e41c04.js",
      "css/appWithAssets-7da09931011a19e41c04.css",
      "js/appWithAssets-7da09931011a19e41c04.js"
    ]
  },
  "publicPath": "http://localhost:3000/",
  "assets": [
    {
      "name": "js/commons-7da09931011a19e41c04.js",
      "path": "js/commons-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/commons-7da09931011a19e41c04.js"
    },
    {
      "name": "js/vendors-7da09931011a19e41c04.js",
      "path": "js/vendors-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/vendors-7da09931011a19e41c04.js"
    },
    {
      "name": "js/2-7da09931011a19e41c04.js",
      "path": "js/2-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/2-7da09931011a19e41c04.js"
    },
    {
      "name": "js/app1-7da09931011a19e41c04.js",
      "path": "js/app1-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/app1-7da09931011a19e41c04.js"
    },
    {
      "name": "css/appWithAssets-7da09931011a19e41c04.css",
      "path": "css/appWithAssets-7da09931011a19e41c04.css",
      "publicPath": "http://localhost:3000/css/appWithAssets-7da09931011a19e41c04.css"
    },
    {
      "name": "js/appWithAssets-7da09931011a19e41c04.js",
      "path": "js/appWithAssets-7da09931011a19e41c04.js",
      "publicPath": "http://localhost:3000/js/appWithAssets-7da09931011a19e41c04.js"
    },
    {
      "name": "js/commons-7da09931011a19e41c04.js.gz",
      "path": "js/commons-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/commons-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/2-7da09931011a19e41c04.js.gz",
      "path": "js/2-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/2-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/vendors-7da09931011a19e41c04.js.gz",
      "path": "js/vendors-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/vendors-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/app1-7da09931011a19e41c04.js.gz",
      "path": "js/app1-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/app1-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "css/appWithAssets-7da09931011a19e41c04.css.gz",
      "path": "css/appWithAssets-7da09931011a19e41c04.css.gz",
      "publicPath": "http://localhost:3000/css/appWithAssets-7da09931011a19e41c04.css.gz"
    },
    {
      "name": "js/appWithAssets-7da09931011a19e41c04.js.gz",
      "path": "js/appWithAssets-7da09931011a19e41c04.js.gz",
      "publicPath": "http://localhost:3000/js/appWithAssets-7da09931011a19e41c04.js.gz"
    },
    {
      "name": "js/2-7da09931011a19e41c04.js.br",
      "path": "js/2-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/2-7da09931011a19e41c04.js.br"
    },
    {
      "name": "js/commons-7da09931011a19e41c04.js.br",
      "path": "js/commons-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/commons-7da09931011a19e41c04.js.br"
    },
    {
      "name": "css/appWithAssets-7da09931011a19e41c04.css.br",
      "path": "css/appWithAssets-7da09931011a19e41c04.css.br",
      "publicPath": "http://localhost:3000/css/appWithAssets-7da09931011a19e41c04.css.br"
    },
    {
      "name": "js/app1-7da09931011a19e41c04.js.br",
      "path": "js/app1-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/app1-7da09931011a19e41c04.js.br"
    },
    {
      "name": "js/vendors-7da09931011a19e41c04.js.br",
      "path": "js/vendors-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/vendors-7da09931011a19e41c04.js.br"
    },
    {
      "name": "js/appWithAssets-7da09931011a19e41c04.js.br",
      "path": "js/appWithAssets-7da09931011a19e41c04.js.br",
      "publicPath": "http://localhost:3000/js/appWithAssets-7da09931011a19e41c04.js.br"
    }
  ]
}

This structure handle files who are not in chunkGroup, so we have bundled apps and rest of the assets. The choice of taking compressed assets will be made by upstream program (ex: django-webpack-loader).

What do you think about that ?

[jonalxh]

You're right, I'm compressing files in the same way, in this case we would talk about the same initial idea so that from the django-webpack-loader we can define what type of file is going to be used, be it .br or .gz

[melck]

@owais Any thoughts on this subject ?

[owais]

The proposal looks good to me. What would the template and python loader interface be to load different files? Would it change in any way?

[melck]

@owais Yes it would change template and python loader. It's clearly a breaking change but it will allow better selection of assets. In my example, webpack compress all files but in real life it will only compress files who have a good ratio and threshold.

Changes for python loader behavior when status done :

• Read chunks entry in json file ;
• For each asset in each chunk name, check asset name in assets ;
• If exist, it load path, publicPath, integrity, etc. for asset.

We can add a option for python loader to priorise selection of compresssed asset. After python loader get list of assets of each chunk, it will look if compressed asset entry exist in asset entry. At first result found, we stop search and use asset details to serve file.

For python loader, we need to add a option COMPRESSED_ASSET_PRIORITY who contains a array of compression by priority (ex: [ 'brotli', 'gzip']). This option will be used to find compressed asset.

[owais]

I understand it'll change the implementation. What I meant was if it'll change interface of how the template tags are written or the Python API. If so, I'd like us to discuss and agree on the new interface first. Implementation details can be discussed later as needed.

[owais]

For python loader, we need to add a option COMPRESSED_ASSET_PRIORITY

My first reaction is to avoid something like that and let users pick which format they'd prefer in the template.

[melck]

For python loader, we need to add a option COMPRESSED_ASSET_PRIORITY

I first reaction is to avoid something like that and let users pick which format they'd prefer in the template.

Good idea :+1: