Minimal Reproduction of `webpack.cache.type="filesystem"` error

[davidmurdoch]

I know the README says the filesystem cache is experimental, but it seems like it works sometimes (it only seems to fail in watch mode in one of my very large projects). I've got a minimal reproduction of the issue below.

What would it take to get this to work?

Reproduction

Create the following files (files are below the output examples), and then run yarn webpack twice.

First run succeeds, but with a warning:

~/code/test/cachetest $ yarn webpack
yarn run v1.22.19
$ /home/david/code/test/cachetest/node_modules/.bin/webpack
<w> [webpack.cache.PackFileCacheStrategy] Skipped not serializable cache item 'html-bundler-webpack-plugin|PersistentCache': Unexpected function (pathData, assetInfo) => {
<w>       // the template filename stays the same after changes in watch mode because have not a hash substitution
<w>       if (assetEntryOptions.isTemplate && assetEntryOptions.filename != null) return assetEntryOptions.filename;
<w> 
<w>       let filename = filenameTemplate;
<w> 
<w>       if (isFunction(filenameTemplate)) {
<w>         // clone the pathData object to modify the chunk object w/o side effects in the main compilation
<w>         const pathDataCloned = { ...pathData };
<w>         pathDataCloned.chunk = { ...pathDataCloned.chunk };
<w>         if (originalName) {
<w>           pathDataCloned.chunk.name = originalName;
<w>           pathDataCloned.chunk.runtime = originalName;
<w>         }
<w> 
<w>         // the `filename` property of the `PathData` type should be a source file, but in entry this property not exists
<w>         if (pathDataCloned.filename == null) {
<w>           pathDataCloned.filename = assetEntryOptions.sourceFile;
<w>         }
<w> 
<w>         filename = filenameTemplate(pathDataCloned, assetInfo);
<w>       }
<w> 
<w>       if (assetEntryOptions.publicPath) {
<w>         filename = path.posix.join(assetEntryOptions.publicPath, filename);
<w>       }
<w> 
<w>       return filename;
<w>     }
<w> while serializing webpack/lib/cache/PackFileCacheStrategy.PackContentItems -> /home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/createPersistentCache.js.PersistentCache -> Map { 1 items } -> Object { entry, assets } -> Object { id, name, originalName, filenameTemplate, filename, assetFile, resource, importFile, sourceFile, sourcePath, outputPath, publicPath, library, verbose, isTemplate, isStyle, filenameFn } -> (pathData, assetInfo) => {
<w>       // the template filename stays the same after changes in watch mode because have not a hash substitution
<w>       if (assetEntryOptions.isTemplate && assetEntryOptions.filename != null) return assetEntryOptions.filename;
<w> 
<w>       let filename = filenameTemplate;
<w> 
<w>       if (isFunction(filenameTemplate)) {
<w>         // clone the pathData object to modify the chunk object w/o side effects in the main compilation
<w>         const pathDataCloned = { ...pathData };
<w>         pathDataCloned.chunk = { ...pathDataCloned.chunk };
<w>         if (originalName) {
<w>           pathDataCloned.chunk.name = originalName;
<w>           pathDataCloned.chunk.runtime = originalName;
<w>         }
<w> 
<w>         // the `filename` property of the `PathData` type should be a source file, but in entry this property not exists
<w>         if (pathDataCloned.filename == null) {
<w>           pathDataCloned.filename = assetEntryOptions.sourceFile;
<w>         }
<w> 
<w>         filename = filenameTemplate(pathDataCloned, assetInfo);
<w>       }
<w> 
<w>       if (assetEntryOptions.publicPath) {
<w>         filename = path.posix.join(assetEntryOptions.publicPath, filename);
<w>       }
<w> 
<w>       return filename;
<w>     }
asset main.js 1.2 KiB [emitted] (name: main)
asset index.html 124 bytes [compared for emit] (name: __bundler-plugin-entry__index)
./src/index.html 206 bytes [built] [code generated]
./src/main.js 27 bytes [built] [code generated]
 HTML Bundler Plugin  ▶▶▶ (webpack 5.89.0) compiled successfully in 50 ms
Done in 1.00s.

Second run fails:

~/code/test/cachetest $ yarn webpack
yarn run v1.22.19
$ /home/david/code/test/cachetest/node_modules/.bin/webpack
<w> [webpack.cache.PackFileCacheStrategy] Skipped not serializable cache item 'html-bundler-webpack-plugin|PersistentCache': Unexpected function (pathData, assetInfo) => {
<w>       // the template filename stays the same after changes in watch mode because have not a hash substitution
<w>       if (assetEntryOptions.isTemplate && assetEntryOptions.filename != null) return assetEntryOptions.filename;
<w> 
<w>       let filename = filenameTemplate;
<w> 
<w>       if (isFunction(filenameTemplate)) {
<w>         // clone the pathData object to modify the chunk object w/o side effects in the main compilation
<w>         const pathDataCloned = { ...pathData };
<w>         pathDataCloned.chunk = { ...pathDataCloned.chunk };
<w>         if (originalName) {
<w>           pathDataCloned.chunk.name = originalName;
<w>           pathDataCloned.chunk.runtime = originalName;
<w>         }
<w> 
<w>         // the `filename` property of the `PathData` type should be a source file, but in entry this property not exists
<w>         if (pathDataCloned.filename == null) {
<w>           pathDataCloned.filename = assetEntryOptions.sourceFile;
<w>         }
<w> 
<w>         filename = filenameTemplate(pathDataCloned, assetInfo);
<w>       }
<w> 
<w>       if (assetEntryOptions.publicPath) {
<w>         filename = path.posix.join(assetEntryOptions.publicPath, filename);
<w>       }
<w> 
<w>       return filename;
<w>     }
<w> while serializing webpack/lib/cache/PackFileCacheStrategy.PackContentItems -> /home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/createPersistentCache.js.PersistentCache -> Map { 1 items } -> Object { entry, assets } -> Object { id, name, originalName, filenameTemplate, filename, assetFile, resource, importFile, sourceFile, sourcePath, outputPath, publicPath, library, verbose, isTemplate, isStyle, filenameFn } -> (pathData, assetInfo) => {
<w>       // the template filename stays the same after changes in watch mode because have not a hash substitution
<w>       if (assetEntryOptions.isTemplate && assetEntryOptions.filename != null) return assetEntryOptions.filename;
<w> 
<w>       let filename = filenameTemplate;
<w> 
<w>       if (isFunction(filenameTemplate)) {
<w>         // clone the pathData object to modify the chunk object w/o side effects in the main compilation
<w>         const pathDataCloned = { ...pathData };
<w>         pathDataCloned.chunk = { ...pathDataCloned.chunk };
<w>         if (originalName) {
<w>           pathDataCloned.chunk.name = originalName;
<w>           pathDataCloned.chunk.runtime = originalName;
<w>         }
<w> 
<w>         // the `filename` property of the `PathData` type should be a source file, but in entry this property not exists
<w>         if (pathDataCloned.filename == null) {
<w>           pathDataCloned.filename = assetEntryOptions.sourceFile;
<w>         }
<w> 
<w>         filename = filenameTemplate(pathDataCloned, assetInfo);
<w>       }
<w> 
<w>       if (assetEntryOptions.publicPath) {
<w>         filename = path.posix.join(assetEntryOptions.publicPath, filename);
<w>       }
<w> 
<w>       return filename;
<w>     }
Entrypoint __bundler-plugin-entry__index =
cached modules 233 bytes [cached] 2 modules

ERROR in [entry] [initial]
 html-bundler-webpack-plugin  Failed to execute the function.
Source file: '/home/david/code/test/cachetest/src/index.html'
/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Messages/Exception.js:109
  throw new PluginException(message);
  ^

PluginException: 
 html-bundler-webpack-plugin  Can't resolve src/main.js in the file src/index.html

    at resolveException (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Messages/Exception.js:109:9)
    at Resolver.require (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Resolver.js:262:7)
    at /home/david/code/test/cachetest/src/index.html:1:116
    at Script.runInContext (node:vm:135:12)
    at VMScript.compile (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/VMScript.js:37:35)
    at Plugin.renderModule (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/AssetCompiler.js:1226:27)
    at Plugin.renderManifest (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/AssetCompiler.js:860:26)
    at Hook.eval [as call] (eval at create (/home/david/code/test/cachetest/node_modules/tapable/lib/HookCodeFactory.js:19:10), <anonymous>:7:16)
    at Hook.CALL_DELEGATE [as _call] (/home/david/code/test/cachetest/node_modules/tapable/lib/Hook.js:14:14)
    at Compilation.getRenderManifest (/home/david/code/test/cachetest/node_modules/webpack/lib/Compilation.js:4525:36)
PluginException: 
 html-bundler-webpack-plugin  Failed to execute the function.
Source file: '/home/david/code/test/cachetest/src/index.html'
/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Messages/Exception.js:109
  throw new PluginException(message);
  ^

PluginException: 
 html-bundler-webpack-plugin  Can't resolve src/main.js in the file src/index.html

    at resolveException (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Messages/Exception.js:109:9)
    at Resolver.require (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Resolver.js:262:7)
    at /home/david/code/test/cachetest/src/index.html:1:116
    at Script.runInContext (node:vm:135:12)
    at VMScript.compile (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/VMScript.js:37:35)
    at Plugin.renderModule (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/AssetCompiler.js:1226:27)
    at Plugin.renderManifest (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/AssetCompiler.js:860:26)
    at Hook.eval [as call] (eval at create (/home/david/code/test/cachetest/node_modules/tapable/lib/HookCodeFactory.js:19:10), <anonymous>:7:16)
    at Hook.CALL_DELEGATE [as _call] (/home/david/code/test/cachetest/node_modules/tapable/lib/Hook.js:14:14)
    at Compilation.getRenderManifest (/home/david/code/test/cachetest/node_modules/webpack/lib/Compilation.js:4525:36)
    at executeFunctionException (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/Messages/Exception.js:120:9)
    at VMScript.compile (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/VMScript.js:41:7)
    at Plugin.renderModule (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/AssetCompiler.js:1226:27)
    at Plugin.renderManifest (/home/david/code/test/cachetest/node_modules/html-bundler-webpack-plugin/src/Plugin/AssetCompiler.js:860:26)
    at Hook.eval [as call] (eval at create (/home/david/code/test/cachetest/node_modules/tapable/lib/HookCodeFactory.js:19:10), <anonymous>:7:16)
    at Hook.CALL_DELEGATE [as _call] (/home/david/code/test/cachetest/node_modules/tapable/lib/Hook.js:14:14)
    at Compilation.getRenderManifest (/home/david/code/test/cachetest/node_modules/webpack/lib/Compilation.js:4525:36)
    at /home/david/code/test/cachetest/node_modules/webpack/lib/Compilation.js:4545:22
    at symbolIterator (/home/david/code/test/cachetest/node_modules/neo-async/async.js:3482:9)
    at timesSync (/home/david/code/test/cachetest/node_modules/neo-async/async.js:2297:7)

 HTML Bundler Plugin  ▶▶▶ (webpack 5.89.0) compiled with 1 error in 48 ms
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Here are the files:

// webpack.config.js
const { join } = require("node:path");
const HtmlBundlerPlugin = require('html-bundler-webpack-plugin');

module.exports = {
    mode: "development",
    cache: {
        type: "filesystem"
    },
    plugins: [
        new HtmlBundlerPlugin({
            entry: {
                index: join(__dirname, "src", "index.html"),
            }
        })
    ]
};

<!-- src/index.html -->
<!DOCTYPE html>
<html>
<head>
    <title>My Page</title>
</head>
<body>
    <script src="main.js"></script>
</body>
</html>

// src/main.js
console.log("hello world");

{
  "name": "cache-test",
  "version": "0.0.1",
  "scripts": {
    "build": "webpack"
  },
  "license": "MIT",
  "devDependencies": {
   "html-bundler-webpack-plugin": "^3.4.11",
    "webpack": "^5.89.0",
    "webpack-cli": "^5.1.4"
  }
}

[davidmurdoch]

Changing the filenameFn assignment in AssetEntry.js to the following seems to "fix" it in cases where the original filenameTemplate is not a function:

    const _filenameFn = isFunction(filenameTemplate) ? filenameFn : assetEntryOptions.publicPath ? path.posix.join(assetEntryOptions.publicPath, filenameTemplate) : filenameTemplate;
    entry.filename = _filenameFn;
    assetEntryOptions.filenameFn = _filenameFn;

but I imagine there are better ways to do this.

[webdiscus]

Hello @davidmurdoch,

thanks for the issuer report. The supporting of the filesystem cache is very very ultra complex. I can't promise, but I will try to fix it.

[davidmurdoch]

Let me know if you want some help!

[webdiscus]

@davidmurdoch, your solution works only for your concrete use case, but brake other functionalities.

I found the right place in the code where the issue must be fixed:

• Collection.js -> serialize() -> write(this.data)

The property this.data.<file>.entry.filenameFn must be serialized as a string, but the original Webpack function write() is buggy and can't serialize a function.

I need to write own serializer for an object contained a function.

[webdiscus]

@davidmurdoch

I have fixed the issue, can you please check the version 3.4.12?

[davidmurdoch]

I'll give it a shot tomorrow! The team I'm on is hiring, if your interested: https://grnh.se/3da035b01us

[davidmurdoch]

I've used it all week and have noticed 0 errors! 👏 🎉 🚀