[BUG] parallel-webpack does not work well with webpack 5 cache persistent

[sunchanglong]

Explain the problem

use ('parallel-webpack').run to compile multiple webpack configs with cache filesystem options paralell-webpack run options

{
    watch: false,
    maxRetries: 1,
    stats: false, // defaults to false
    maxConcurrentWorkers: 2 // use 2 workers
}

Expected Behaviour

create .cache directory

Actual Behaviour

does not create .cache directory

Steps to reproduce

set run method options 

{
    watch: false,
    maxRetries: 1,
    stats: false, // defaults to false
    maxConcurrentWorkers: 2 // use 2 workers
}

note: it will create .cache directory if set stats: true

Provide your webpack config

Provide your Environment details

• Node version:

• Operating System:

• webpack version: 5.11.1

• parallel-webpack version: 2.6.0

[wearebear80]

I have the same problem. Yes - persistent cache directory created if i set stats: true , but cache not working - my build time does not change :(

pure webpack build time: 1 min 30 sec webpack + persistent cache build time: 35 sec parallel-webpack (two parallel builds) build time: 1 min 30 sec parallel-webpack + persistent cache (two parallel builds) build time: 1 min 30 sec

I tried to find the problem and i think its cause this JSON.stringify, but i'm not sure

https://github.com/trivago/parallel-webpack/blob/902c471fc838f498e392330a4eb360463f2c8923/src/webpackWorker.js#L205

if i provide pure stats to done callback like:

done(null, stats)

persistent cache working fine, but my tasks does't exit from the process - my task freezes forever Maybe its because webpack cache working with stats in deferred webpack-hook, but its just a guess

I will try to write a similar issue to webpack repo and will attach a link here.

[filmic]

In Webpack 5 compiler must be closed so that low-priority work (like persistent caching) have the opportunity to complete. https://webpack.js.org/migrate/5/#cleanup-the-code https://webpack.js.org/api/node/#run

Wrapping the done() call in webpackWorker.js with compiler.close()solved the problem for me:

compiler.close((closeErr) => {
    if (closeErr) console.error('Close error occured', closeErr);
    cleanup();
    if (disconnected) {
        return;
    }
    done(null, options.stats ? JSON.stringify(stats.toJson(outputOptions), null, 2) : '');
});