Production build hangs at 92%

[melloc01]

Hello,

I think this is probably a bug with this plugin since the dev build works fine and when I run webpack --progress it gets stuck in:

92% chunk asset optimization UglifyJSPlugin

Below, the relevant parts of my webpack.config:

....

module.exports = {
    mode,
    node: { fs: 'empty' },
    entry: {
        ...entries
    },

    output: {
        filename: '[name].js',
        path: path.join(__dirname, '/public/assets/')
    },

    module: {
        rules: [
            {
                test: /\.js?$/,
                exclude: /(node_modules)/,
                use: 'babel-loader'
            },
            {
                test: /\.(scss|css)$/,
                use: [
                    MiniCssExtractPlugin.loader,

                    {
                        loader: 'css-loader'
                    },
                    {
                        loader: 'sass-loader',
                        options: {
                            sourceMap: true
                        }
                    }
                ]
            }
        ]
    },

    plugins: [
        new MiniCssExtractPlugin({
            filename: '[name].css'
        })
    ],

    resolve: {
        alias: {
            core: path.join(__dirname, 'js/core')
        }
    },

    optimization: {
        splitChunks: {
            minSize: 100000,
            chunks: 'initial',
            maxAsyncRequests: Infinity,
            maxInitialRequests: Infinity,
            name: true,
            cacheGroups: {
                styles: {
                    name: 'core',
                    test: /\.(scss|css)$/,
                    chunks: 'all',
                    enforce: true
                },
                vendor: {
                    name: 'vendor',
                    test: function (module) {
                        return /\/node_modules\//.test(module.context)
                    }
                },
                'core': {
                    name: 'core',
                    test: function (module) {
                        return /live-core\//.test(module.context)
                    }
                }
            }
        }
    }
}

Is this enough info? I tried to run with --debug --progress --display-error-details --verbose but nothing else came out

[melloc01]

Well, I post the config in case there is an error on it - if you don't spot an error I can create a repo that reproduces the error,

Thanks in advance :blush:

[melloc01]

Instead of using the build-in module from webpack, when actually use the plugin on my webpack-config.js adding it to the plugins array and in dev mode, it works as expected, productionfails again. Perhaps it's related to webpack.. :thinking:

[melloc01]

Still experimenting...

Using -p seems to be the reason that I get stuck. (maybe it's the only way to activate the plugin?)

Using https://github.com/pingyuanChen/webpack-uglify-js-plugin works on -p mode, it was the workaround to get going for now

[alexander-akait]

@melloc01 can you create minimum reproducible test repo?

[alexander-akait]

@melloc01 Friendly ping :+1:

[melloc01]

Thanks @evilebottnawi Did not forget - just overloaded with other upgrade issues. HMR now :|

Will save some time on the weekend to set up the repo

News: someone on Gitter is having the same issue

[zlk89]

Same here, and I get out of memory error after a while:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

This error happens to me every time I tried to use UglifyJSPlugin.

My dependencies:

{
    "autodll-webpack-plugin": ">=0.3.9",
    "autoprefixer": ">=8.2.0",
    "babel-loader": ">=8.0.0-beta.2",
    "babel-plugin-import": ">=1.7.0",
    "circular-dependency-plugin": ">=5.0.1",
    "compression-webpack-plugin": ">=1.1.11",
    "css-loader": ">=0.28.11",
    "file-loader": ">=1.1.11",
    "font-loader": ">=0.1.2",
    "happypack": ">=5.0.0-beta.3",
    "html-webpack-plugin": ">=3.2.0",
    "lodash": ">=4.17.5",
    "null-loader": ">=0.1.1",
    "postcss-loader": ">=2.1.3",
    "preload-image-loader": ">=1.0.0",
    "react-hot-loader": ">=4.0.1",
    "resolve-url-loader": ">=2.3.0",
    "sass-loader": ">=6.0.7",
    "stats-webpack-plugin": ">=0.6.2",
    "style-loader": ">=0.20.3",
    "uglifyjs-webpack-plugin": ">=1.2.4",
    "webpack": ">=4.5.0",
    "webpack-cli": ">=2.0.14"
  }

[alexander-akait]

@zlk89 do you use latest version?

[alexander-akait]

@zlk89 Also please create new issue, your problem is other

[erickwilder]

I just upgraded to webpack 4 (4.5.0) and I'm experiencing the same. It takes ages to finish the build process as it gets stuck on 92%.

[alexander-akait]

@erickwilder Can you crete minimum reproducible test repo?

[erickwilder]

My issue was apparently caused by the compress: true option of uglify - which is the default. Setting a custom uglify plugin instance overriding this configuration under the plugins list had no effect to change the compilation behavior.

I did solve it by moving uglify from the plugins to the optimization.minimizer list.

optimization.minimizer: [ new UglifyJsPlugin(...custom options) ]

Unfortunately this was the work of some digging through the code and other people's setup - I didn't find clear documentation about it.

[alexander-akait]

@erickwilder in webpack@4 you don't use plugins for minification, use minimizer property. If you use plugin field you have 2 instance of plugin. But will be great minimum reproducible test repo. In theory it is don't hangs.

[melloc01]

@evilebottnawi actually, if you specify the minimize property you just overwrite the default (built-in) UglifyJS as seen here: https://survivejs.com/webpack/optimizing/minifying/

I actually set up a dummy repo that mimics the file structure (w/ dummy modules) of the original project but unfortunately it didn't hang, so it's not reproducible using dummy modules

I'm up for a call if you wanna dig into it @evilebottnawi, it's private so I can't just put on gh =\

[alexander-akait]

@melloc01 can you add coded in uglify plugin here https://github.com/webpack-contrib/uglifyjs-webpack-plugin/blob/master/src/index.js#L184

console.log(tasks[index]); and identify hwat task is not completed?

[melloc01]

sure - done it.

I'm using exactly this source: https://gist.github.com/melloc01/4c34922b2e31599674a7ddff0aaa7d2d#file-uglify-index-js-L138

Output:

$ webpack --config webpack-prod.config.js --progress -p

Plugin init...
 60% building modules 420/455 modules 35 active …ore/services/config/dashboard.config.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 64% building modules 457/477 modules 20 active …modules/react-bootstrap/es/Jumbotron.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 65% building modules 475/517 modules 42 active …js/modules/main/monitoring/ui/alerts.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 67% building modules 539/564 modules 25 active …modules/react-bootstrap/es/MediaLeft.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2406/2450 modules 44 active …-core/components/header/ui/main/main.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2415/2453 modules 38 active …tcss/node_modules/lodash/_mapToArray.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2424/2455 modules 31 active …/modules/admin/components/roles-form.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2425/2459 modules 34 active …/node_modules/babel-standalone/babel.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2428/2463 modules 35 active …/node_modules/sass.js/dist/sass.sync.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2431/2470 modules 39 active …dules/admin/extensions/ui/pipes/form.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2433/2488 modules 55 active …modules/core-js/modules/_typed-array.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2436/2490 modules 54 active …/node_modules/lodash/_listCacheClear.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2440/2495 modules 55 active …s/live/webapp/js/live-core/lib/types.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2445/2503 modules 58 active …olor/node_modules/lodash/_hashDelete.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2447/2504 modules 57 active …ve-core/components/ui/dashboard-card.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2464/2510 modules 46 active …core-js/modules/_inherit-if-required.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2465/2510 modules 45 active …core-js/modules/_inherit-if-required.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 68% building modules 2467/2517 modules 50 active …components/header/ui/main/my-account.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2495/2524 modules 29 active …es/enquire.js/src/MediaQueryDispatch.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2497/2524 modules 27 active …es/enquire.js/src/MediaQueryDispatch.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2515/2545 modules 30 active …or/node_modules/lodash/_nativeCreate.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2610/2628 modules 18 active …app/node_modules/miller-rabin/lib/mr.js
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2710/2713 modules 3 active …e_modules/jquery-ui/themes/base/all.css 
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 69% building modules 2712/2713 modules 1 active …ael/Projects/live/webapp/sass/live.scss
After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []

After chunks reduce, before runTasks... tasks len= 0

InsideRunTask... results= []
 92% chunk asset optimization UglifyJSPlugin               
After chunks reduce, before runTasks... tasks len= 7
filename = vendor.js

[alexander-akait]

@melloc01 very intresting, you always have zero tasks :confused: You enable parallel? If yes, can you disable and check again? If no can you enable and check again? :smile:

[melloc01]

Anything else than the filename to help debugging @evilebottnawi?

[melloc01]

After a while:

Same thing with parallel: false @evilebottnawi

<--- Last few GCs --->

[2509:0x2be3950]   115719 ms: Mark-sweep 1407.9 (1504.7) -> 1405.2 (1506.7) MB, 1204.8 / 0.0 ms  allocation failure GC in old space requested
[2509:0x2be3950]   117133 ms: Mark-sweep 1405.2 (1506.7) -> 1405.2 (1472.7) MB, 1413.8 / 0.0 ms  last resort GC in old space requested
[2509:0x2be3950]   118372 ms: Mark-sweep 1405.2 (1472.7) -> 1405.2 (1468.7) MB, 1237.9 / 0.0 ms  last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x34e23ad25501 <JSObject>
    1: push(this=0x34d6cddcc679 <JSArray[50855]>)
    2: reduce_vars [0x322b08a822d1 <undefined>:~8085] [pc=0x2244554c2807](this=0x3d7535065621 <AST_SymbolRef map = 0x7dfe33070b9>,tw=0x3716375ac559 <TreeWalker map = 0x7dfe330cde1>,descend=0x1c8762082991 <JSFunction noop (sfi = 0x1d796426e2a1)>,compressor=0x2de51d44eea1 <Compressor map = 0x2a0097b73f31>)
    3: visit [0x322b08a822d1 <undefined>:~...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x11f155c [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewUninitializedFixedArray(int) [node]
 6: 0xdf28df [node]
 7: 0xe05ff5 [node]
 8: v8::internal::JSObject::AddDataElement(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow) [node]
 9: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
10: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
11: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
12: 0x2244548842fd
Aborted (core dumped)
error Command failed with exit code 134.

[melloc01]

Config:

        minimizer: [
            new UglifyJsPlugin({
                cache: false,
               parallel: false
            }),
            new OptimizeCSSAssetsPlugin({})
        ]

[alexander-akait]

@melloc01 can you idntify after what file it is happens? Just add console.log(file); here https://github.com/webpack-contrib/uglifyjs-webpack-plugin/blob/master/src/index.js#L184. Also add console.log(this.maxConcurrentWorkers) here https://github.com/webpack-contrib/uglifyjs-webpack-plugin/blob/master/src/uglify/index.js#L19. Thanks for help!

[melloc01]

I'm back... debugging:

parallel=false

Plugin init...
 62% building modules 434/468 modules 34 active …es/react-bootstrap/es/DropdownButton.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 66% building modules 518/554 modules 36 active …t-dom/lib/renderSubtreeIntoContainer.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 597/616 modules 19 active …les/babel-runtime/core-js/array/from.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 691/707 modules 16 active …odules/core-js/library/fn/array/from.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2406/2421 modules 15 active …app/node_modules/rc-slider/lib/utils.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2410/2423 modules 13 active …/node_modules/react-slick/lib/slider.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2440/2465 modules 25 active …modules/core-js/modules/_typed-array.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2458/2493 modules 35 active …ode_modules/core-js/modules/_classof.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2458/2497 modules 39 active …ode_modules/core-js/modules/_wks-ext.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2463/2512 modules 49 active …e-core/components/ui/FullScreenModal.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2464/2512 modules 48 active …e-core/components/ui/FullScreenModal.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2467/2515 modules 48 active …modules/core-js/modules/_iter-create.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2470/2522 modules 52 active …-core/components/forms/ui/table-form.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2481/2523 modules 42 active …ode_modules/simplebar/dist/simplebar.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2489/2523 modules 34 active …ode_modules/simplebar/dist/simplebar.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2495/2523 modules 28 active …ode_modules/simplebar/dist/simplebar.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2495/2524 modules 29 active …p/node_modules/delegate/src/delegate.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2502/2526 modules 24 active …_modules/core-js/modules/_object-dps.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2503/2528 modules 25 active …de_modules/core-js/modules/_to-index.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2504/2528 modules 24 active …de_modules/core-js/modules/_to-index.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2570/2603 modules 33 active …de_modules/public-encrypt/withPublic.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2580/2607 modules 27 active …_modules/pbkdf2/lib/default-encoding.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2582/2607 modules 25 active …_modules/pbkdf2/lib/default-encoding.js
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2712/2713 modules 1 active …ael/Projects/live/webapp/sass/live.scss 
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 0
 92% chunk asset optimization UglifyJSPlugin               
maxConcurrentWorkers 0

After chunks reduce, before runTasks... tasks len= 7
filename =  vendor.js

<--- Last few GCs --->

[20574:0x382c950]   100951 ms: Mark-sweep 1382.4 (1498.1) -> 1382.3 (1499.1) MB, 1213.9 / 0.0 ms  allocation failure GC in old space requested
[20574:0x382c950]   102171 ms: Mark-sweep 1382.3 (1499.1) -> 1382.3 (1458.1) MB, 1219.6 / 0.0 ms  last resort GC in old space requested
[20574:0x382c950]   103401 ms: Mark-sweep 1382.3 (1458.1) -> 1382.3 (1454.6) MB, 1229.4 / 0.0 ms  last resort GC in old space requested
...

parallel=true

$ webpack --config webpack-prod.config.js --progress -p

Plugin init...
 67% building modules 579/606 modules 27 active …e/webapp/node_modules/isobject/index.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 918/940 modules 22 active …e-js/library/modules/_array-includes.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2457/2483 modules 26 active …modules/velocity-animate/velocity.ui.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2459/2485 modules 26 active …les/rc-util/lib/Dom/addEventListener.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2459/2486 modules 27 active …/node_modules/react-slick/lib/slider.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2461/2490 modules 29 active …/node_modules/babel-standalone/babel.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2463/2494 modules 31 active …/node_modules/sass.js/dist/sass.sync.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2463/2497 modules 34 active …_modules/core-js/modules/_wks-define.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2463/2499 modules 36 active …de_modules/core-js/modules/_is-array.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2464/2513 modules 49 active …_modules/core-js/modules/_collection.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2467/2515 modules 48 active …modules/core-js/modules/_typed-array.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2467/2517 modules 50 active …odules/core-js/modules/_typed-buffer.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2471/2517 modules 46 active …odules/core-js/modules/_typed-buffer.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2474/2519 modules 45 active …ebapp/node_modules/select/src/select.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 68% building modules 2476/2519 modules 43 active …ebapp/node_modules/select/src/select.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2490/2520 modules 30 active …js/library/fn/object/define-property.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2492/2520 modules 28 active …js/library/fn/object/define-property.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2503/2531 modules 28 active …de_modules/core-js/modules/_to-index.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2507/2534 modules 27 active …modules/react-responsive-mixin/index.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2558/2586 modules 28 active …ebapp/node_modules/cipher-base/index.js
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2565/2589 modules 24 active …les/browserify-sign/browser/curves.json
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2568/2589 modules 21 active …les/browserify-sign/browser/curves.json
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 69% building modules 2712/2713 modules 1 active …ael/Projects/live/webapp/sass/live.scss 
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0

maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 0
 92% chunk asset optimization UglifyJSPlugin               
maxConcurrentWorkers 3

After chunks reduce, before runTasks... tasks len= 7
filename =  vendor.js

...(not dead yet)

[melloc01]

Btw, I updated the gist. Look for .log('==========file======') it never reaches that point.

[alexander-akait]

@melloc01 can you rewrite https://github.com/webpack-contrib/uglifyjs-webpack-plugin/blob/master/src/uglify/worker.js#L3 on:

module.exports = (options, callback) => {
  try {
    // 'use strict' => this === undefined (Clean Scope)
    // Safer for possible security issues, albeit not critical at all here
    // eslint-disable-next-line no-new-func, no-param-reassign
    options = new Function(`'use strict'\nreturn ${options}`)();
    const minified = minify(options); 

    callback(minified.error ? 'Error' : null, minified);
  } catch (errors) {
    callback(errors);
  }
};

And run in parallels mode?

[melloc01]

@evilebottnawi sorry about that last comment, it wasn't right.

Actually, I am modifying the babelified version of the plugin, so minify wasn't defined, got an error from the plugin.

on babelified version it is minify2.default

[alexander-akait]

@melloc01 yep, rewrite using minify2.default and other babel stuff with save logic as above

[melloc01]

Same outcome - stuck at 92% until core dump Actually, I'm free now for faster interactions :+1:

I also logged the minified object and saw that in the code there were ModuleConcatenation bailout repeating a lot. Disabling it to see if there is some change.

[alexander-akait]

@melloc01 can you disable ModuleConcatenationPlugin and check again?

[melloc01]

       namedModules: false,
        noEmitOnErrors: false, // NoEmitOnErrorsPlugin
        concatenateModules: false, // ModuleConcatenationPlugin

It seems that it is getting stuck in somewhere else @evilebottnawi ... I changed the worker code to this:

'use strict'

var _minify = require('./minify')

var _minify2 = _interopRequireDefault(_minify)

function _interopRequireDefault (obj) {
    return obj && obj.__esModule ? obj : { default: obj }
}

module.exports = function (options, callback) {
    try {
        // 'use strict' => this === undefined (Clean Scope)
        // Safer for possible security issues, albeit not critical at all here
        // eslint-disable-next-line no-new-func, no-param-reassign
        options = new Function(`'use strict'\nreturn ${options}`)()
        console.warn('')
        console.warn('options =', options.file)
        const minified = _minify2.default(options)

        console.warn('after options')
        console.warn('')

        /*
          Options:
          [ 'file',
  'input',
  'inputSourceMap',
  'commentsFile',
  'extractComments',
  'uglifyOptions' ]

        */

        callback(minified.error ? 'Error' : null, minified)
    } catch (errors) {
        callback(errors)
    }'use strict'

var _minify = require('./minify')

var _minify2 = _interopRequireDefault(_minify)

function _interopRequireDefault (obj) {
    return obj && obj.__esModule ? obj : { default: obj }
}

module.exports = function (options, callback) {
    try {
        // 'use strict' => this === undefined (Clean Scope)
        // Safer for possible security issues, albeit not critical at all here
        // eslint-disable-next-line no-new-func, no-param-reassign
        options = new Function(`'use strict'\nreturn ${options}`)()
        console.warn('')
        console.warn('options =', options.file)
        const minified = _minify2.default(options)

        console.warn('after options')
        console.warn('')

        /*
          Options:
          [ 'file',
  'input',
  'inputSourceMap',
  'commentsFile',
  'extractComments',
  'uglifyOptions' ]

        */

        callback(minified.error ? 'Error' : null, minified)
    } catch (errors) {
        callback(errors)
    }
}

}

and the output:

After chunks reduce, before runTasks... tasks len= 0
 92% chunk asset optimization UglifyJSPlugin               
After chunks reduce, before runTasks... tasks len= 7
filename =  vendor.js

options = admin.js

options = live.js

options = vendor.js
after options

options = dashboard.js
after options

after options

options = main.js
after options

It's stuck consuming memory right now

[melloc01]

Let me try this will parallel=false

[melloc01]

it won't use any workers of course :man_facepalming:

I'm trying some stuff here. If you come up with new ideas I'll be glad to tweak things here :+1:

[alexander-akait]

@melloc01 can you provide sizes of assets here https://github.com/webpack-contrib/uglifyjs-webpack-plugin/blob/master/src/index.js#L122:

console.log(source.length);

[melloc01]

source.length 18748086
source.length 10268452
source.length 572771
source.length 517471
source.length 657436
source.length 511557
source.length 135605

[alexander-akait]

@melloc01 Can you try this branch https://github.com/webpack-contrib/uglifyjs-webpack-plugin/pull/276, remove uglify-webpack-plugin from node_modules, git clone this repo, npm i, when npm build and test build

[melloc01]

Doing it - not completely sure how it'll help (since it gets stucks even without SourceMaps) but it is for sure an improvement.

[melloc01]

Output on #276

➜  uglifyjs-webpack-plugin git:(perf-reduce-memory-usage) 

➜  webapp git:(wip/webpack4) ✗ yarn run build
yarn run v1.3.2
$ webpack --config webpack-prod.config.js --progress -p
 92% chunk asset optimization UglifyJSPlugin
<--- Last few GCs --->

[6791:0x2a72a20]    47493 ms: Mark-sweep 1382.6 (1461.7) -> 1382.5 (1463.2) MB, 1116.4 / 0.0 ms  allocat
ion failure GC in old space requested
[6791:0x2a72a20]    48628 ms: Mark-sweep 1382.5 (1463.2) -> 1382.4 (1424.2) MB, 1134.4 / 0.0 ms  last re
sort GC in old space requested
[6791:0x2a72a20]    49751 ms: Mark-sweep 1382.4 (1424.2) -> 1382.4 (1424.2) MB, 1122.7 / 0.0 ms  last re
sort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x17b4bcca5501 <JSObject>
    1: push(this=0x3297e6003199 <JSArray[50855]>)
    2: reduce_vars [0x3f88c08022d1 <undefined>:~8085] [pc=0x13b05d804147](this=0x1ac4aae6a839 <AST_Symbo
lRef map = 0x245b3966aa01>,tw=0xb9cb8502291 <TreeWalker map = 0x245b39669771>,descend=0x1703744a7829 <JS
Function noop (sfi = 0x37461f61bb1)>,compressor=0x1839f239e409 <Compressor map = 0x3352ad49b641>)
    3: visit [0x3f88c08022d1 <undefined>:~...

...

[alexander-akait]

@melloc01 maybe you can create minimum reproducible test repo or send me temporary invite?

[iugo]

I have a same issue.

When the problem occurs, the CPU load decreases(< 1%), but webpack does not report an error. Only occurs on the WSL platform. There is no such issue on Windows and macOS.

I think this kind of conflict caused by some code module.

[alexander-akait]

@iugo can you create minimum reproducible test repo?

[iugo]

@evilebottnawi I try to reproduce with minimal code. But it's hard. It seems that there is only a lot of code to reproduce the problem.

92% chunk asset optimization UglifyJSPlugin
After chunks reduce, before runTasks... tasks len= 0                                                    92% chunk asset optimization UglifyJSPluginAfter chunks reduce, before runTasks... tasks len= 77
filename =  ExChange~QueryTransfer~ShopCoinInfo~Transfer~accounting-vouchers~auditor-voucher~balance-sheet~cashi~9e25abfe.354b276e73703fdb23a7.chunk.js

[alexander-akait]

@iugo look we have memory leak (maybe in uglify-es), but i can't detect place without reproducible repo :disappointed:

[iugo]

Finally finished: Time: 500946ms

The general time is 40s~50s.

[alexander-akait]

@iugo do you use parallel options?

[iugo]

@evilebottnawi Yes.

new UglifyJsPlugin({
  test: /\.js($|\?)/i,
  include: /\/src/,
  parallel: true,
  sourceMap: false,
}),

[alexander-akait]

@iugo if you can create minimum reproducible test repo, it will be great, maybe we have memory leak

[iugo]

@evilebottnawi

Ubuntu 16.04.4 (WSL Windows 10 1803) Node v8.11.1 yarn 1.5.1

https://drive.google.com/file/d/1rRq48YpIc5zqUhlP_c5X7N4KhpQsbKPf/view?usp=sharing

yarn run build

[alexander-akait]

@iugo Very Very Very thanks!

[alexander-akait]

@iugo Initial build:

Hash: 50f9ade2d7b637a5a9ae                                                              
Version: webpack 4.2.0
Time: 51763ms
Built at: 2018-4-16 12:31:20

Second build:

Hash: 50f9ade2d7b637a5a9ae                                                              
Version: webpack 4.2.0
Time: 25236ms
Built at: 2018-4-16 12:44:54

Can you show hardware list (cpu/memory/ssd/hdd?)?

[alexander-akait]

@iugo you stack

UGLIFY vendors~ExChange~QueryTransfer~Transfer~accounts-ledger~accounts-manage~cashier-create-order-temp~ca~32d43d74.5a0de7f4bb74bb13ee35.chunk.js: 1143.496ms
UGLIFY vendors~ExChange~QueryTransfer~Transfer~accounts-manage~cashier-create-orders~coupon-verification~co~50856029.22913466754cc6fbf7f3.chunk.js: 1372.387ms
UGLIFY vendors~ExChange~QueryTransfer~Transfer~accounts-manage~black-page~cashier-create-orders~coupon-clas~38f46356.fd15c81dc573e8d926c2.chunk.js: 3168.370ms
UGLIFY vendors~Transfer~coupon-classes~purchase-backs~purchasing-create.e10cd0a117ad22a19883.chunk.js: 2699.113ms
UGLIFY coupons-grant.d2d9046bb4f8b7868a50.chunk.js: 3848.956ms
UGLIFY log.15c5a483c9a1c480d0f6.chunk.js: 24.688ms
UGLIFY vendors~ExChange~QueryTransfer~Transfer~accounts-manage~black-page~cashier-create-order-temp~cashier~57c1b378.8fe40118bb2696be3f67.chunk.js: 5930.115ms
UGLIFY ExChange~cashier-create-order-temp~cashier-create-orders~create-refunds~refunds.5457f678a180eb14b82f.chunk.js: 591.601ms
UGLIFY accounts-ledger.e9c2ede717a69dd6a00e.chunk.js: 387.038ms
UGLIFY create-coupons.ad124f9fb2c28c1cd98a.chunk.js: 4803.816ms
UGLIFY roles-auth.3cf05d5d63a751735049.chunk.js: 2881.869ms
UGLIFY accounts-manage.fda39c0f13c3725c978b.chunk.js: 5752.888ms
UGLIFY vendors~ExChange~QueryTransfer~Transfer~accounts-ledger~accounts-manage~black-page~cashier-create-or~b8519a2e.c3a396d22b81ea24acc6.chunk.js: 9375.177ms
UGLIFY coupon-classes.15a77577fa13560301d4.chunk.js: 583.937ms
UGLIFY black-page~log.15490dfce94c8df70360.chunk.js: 626.554ms
UGLIFY coupon-verification.4a9ce07afcf97c1f4d6e.chunk.js: 3580.749ms
UGLIFY ExChange.56f724182a3006231671.chunk.js: 6181.313ms
UGLIFY purchasing-create.afea65ea406c4c0f9ac4.chunk.js: 6531.311ms
UGLIFY create-refunds.3a1a5b16cfadc1401340.chunk.js: 5246.948ms
UGLIFY purchase-backs.c16ad5a0f58917fd6276.chunk.js: 6218.182ms
UGLIFY paymode-management.31ca4772ff769c49e5ad.chunk.js: 6090.606ms
UGLIFY black-page.81958dbf32b156ed447f.chunk.js: 236.184ms
UGLIFY roles-list.58db7b0cdd63b65f94ba.chunk.js: 4308.598ms
UGLIFY QueryTransfer.6454e56ac3d6597c1b31.chunk.js: 4214.266ms
UGLIFY ExChange~QueryTransfer~Transfer~accounts-manage~cashier-create-orders~coupon-verification~coupons-gr~a67d4740.b15b83375b6c9c4020ca.chunk.js: 1646.734ms
UGLIFY vendors-api.b6b34b75521b85b38ec5.chunk.js: 10.390ms
UGLIFY employeesManage.39be7ea460ad41c31d7d.chunk.js: 4764.902ms
UGLIFY refunds.97b6b74636e793d867b6.chunk.js: 4354.618ms
UGLIFY cashier-create-order-temp.07d40e1388b3a83fd3d6.chunk.js: 215.237ms
UGLIFY departments.eefc3dfccfed45aab790.chunk.js: 5392.898ms
UGLIFY ExChange~QueryTransfer~Transfer~cashier-create-orders~coupon-verification~coupons-grant~create-refun~e3a139ad.ac1b852a951200b5c08f.chunk.js: 276.284ms
UGLIFY warehouse-api.9670826a1333974a58d6.chunk.js: 16.890ms
UGLIFY purchasing-checking.391201bba540b515a9fe.chunk.js: 3632.985ms
UGLIFY warehouse.83320a35fe2af2f2ff51.chunk.js: 4372.242ms
UGLIFY 35.5b7dc84313c2108c6c7a.chunk.js: 144.959ms
UGLIFY cashier-create-orders.7500b5abad985ebf3644.chunk.js: 5351.029ms
UGLIFY orders.819ce3b46063907e9e8e.chunk.js: 69.151ms
UGLIFY Transfer.fa7d7ae323ce6b63e250.chunk.js: 4609.111ms
UGLIFY 34.9d48869bbdcf2bcb3b28.chunk.js: 138.515ms
UGLIFY main.50f9a.bundle.js: 64.173ms       

Looks like uglify very very slow :disappointed:

[alexander-akait]

@melloc01 can you add lines:

  console.time('UGLIFY ' + file);
  var _uglify$minify = _uglifyEs2.default.minify({ [file]: input }, uglifyOptions),
      error = _uglify$minify.error,
      map = _uglify$minify.map,
      code = _uglify$minify.code,
      warnings = _uglify$minify.warnings;
  console.timeEnd('UGLIFY ' + file);

in dist/uglify/minify.js and put your output as above here?