tettusud / merge-jsons-webpack-plugin

This plugin is used to merge json files into single json file,using glob or file names
https://npmjs.com/package/merge-jsons-webpack-plugin
Apache License 2.0
36 stars 20 forks source link

Plugin causes Webpack@3.3.0 compiler to compile twice. #18

Closed bridzius closed 6 years ago

bridzius commented 7 years ago

Calling the compiler from nodeJS.

  const compiler = webpack(config);
  compiler.run((err, stats) => {
    if (err || stats.hasErrors()) {
       console.log("error when bundling");
       reject(err);
     }
     console.log("File bundling finished.");
     resolve();
   });

If using webpack@3.3.0 - the merge plugin causes the following output

(node:11694) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.
(node:11694) DeprecationWarning: Chunk.modules is deprecated. Use Chunk.getNumberOfModules/mapModules/forEachModule/containsModule instead.
MergetJsonsWebpackPlugin emit...
File bundling finished.
Tarball created - Webserver Configuration added.
File bundling finished.

Removing merge plugin produces following output:

(node:11655) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.

(node:11655) DeprecationWarning: Chunk.modules is deprecated. Use Chunk.getNumberOfModules/mapModules/forEachModule/containsModule instead.
File bundling finished.
Tarball created - Webserver Configuration added.
tettusud commented 7 years ago

Could you please attach whole error log,looks like one of the Json file you are trying to merge is mal formed. Possibly the one with the below content 21_13_58_23","process":"Deploy_FDP_private_CM By the way I believe you are using the latest version

----- Reply message ----- From: "Jurgis Bridžius" notifications@github.com To: "tettusud/merge-jsons-webpack-plugin" merge-jsons-webpack-plugin@noreply.github.com Cc: "Subscribed" subscribed@noreply.github.com Subject: [tettusud/merge-jsons-webpack-plugin] Plugin causes Webpack@3.3.0 compiler to compile twice. (#18) Date: Fri, Jul 21, 2017 10:35 PM

Calling the compiler from nodeJS. const compiler = webpack(config); compiler.run((err, stats) => { if (err || stats.hasErrors()) { console.log("error when bundling"); reject(err); } console.log("File bundling finished."); resolve(); });

If using webpack@3.3.0 - the merge plugin causes the following output (node:11694) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56 parseQuery() will be replaced with getOptions() in the next major version of loader-utils. (node:11694) DeprecationWarning: Chunk.modules is deprecated. Use Chunk.getNumberOfModules/mapModules/forEachModule/containsModule instead. MergetJsonsWebpackPlugin emit... File bundling finished. Tarball created - Webserver Configuration added. 21_13_58_23","process":"Deploy_FDP_private_CM","environment":"","subpath":"nib_private"} File bundling finished.

Removing merge plugin produces following output: (node:11655) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see https://github.com/webpack/loader-utils/issues/56 parseQuery() will be replaced with getOptions() in the next major version of loader-utils.

(node:11655) DeprecationWarning: Chunk.modules is deprecated. Use Chunk.getNumberOfModules/mapModules/forEachModule/containsModule instead. File bundling finished. Tarball created - Webserver Configuration added.

—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/tettusud/merge-jsons-webpack-plugin","title":"tettusud/merge-jsons-webpack-plugin","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/tettusud/merge-jsons-webpack-plugin"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"Plugin causes Webpack@3.3.0 compiler to compile twice. (#18)"}],"action":{"name":"View Issue","url":"https://github.com/tettusud/merge-jsons-webpack-plugin/issues/18"}}}

bridzius commented 7 years ago

@tettusud - there doesn't seem to be any error thrown. This is, basically, what we get when we run with the default compiler config (stats ignored, errors shown). The strangeness is, that if you were to run via cli - it builds fine.

tettusud commented 7 years ago

@bridzius Unfortunately am not able to reproduce it , I tried all possibilities, if you can send me detailed steps how to reproduce, may be it will be useful to debug. By the way I dont get you "default compiler config", what do you mean exactly?

jmesa-sistel commented 6 years ago

I have the same problem. You can reproduce it with an empty jhipster project. MergetJsonsWebpackPlugin compilation started... MergetJsonsWebpackPlugin compilation completed... 288ms asset optimization 95% emittingMergetJsonsWebpackPlugin emit starts... MergetJsonsWebpackPlugin emit completed... 219ms emitting 1577 modules webpack: Compiled successfully. MergetJsonsWebpackPlugin emit starts... MergetJsonsWebpackPlugin emit completed... 1577 modules webpack: Compiled successfully.

tettusud commented 6 years ago

Could you paste your webpack.config.js By any chance are you using the plugin twice in the Config file ?

Sent from my iPhone

On 24 Oct 2017, at 1:42 AM, Julián Mesa Llopis notifications@github.com wrote:

I have the same problem. You can reproduce it with an empty jhipster project.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

jmesa-sistel commented 6 years ago

You can see the webpack files here: https://github.com/jhipster/generator-jhipster/tree/master/generators/client/templates/angular/webpack Webpack is not recompiling twice but send more signals to VSCode and to the web application, so the webapp is reloaded 2 or 3 times and hot module reload is not effective.

tettusud commented 6 years ago

@jmesa-sistel , in the application you generated usin jhipster can you tell me if you got more than one instance of the plugin initialized inside the webpack.config.js?

jmesa-sistel commented 6 years ago

@tettusud Only once, I am not sure but I think the problem is that you are adding the new generated json files in fileDependencies instead of assets, so the webdev detect a change in files and send a signal telling that something is changed.

output without plugin:

webpack: Compiling...
571ms building modules
30ms sealing
2ms optimizing
1ms basic module optimization
18ms module optimization
2ms advanced module optimization
19ms basic chunk optimization
0ms chunk optimization
1ms advanced chunk optimization
1ms module and chunk tree optimization
1ms chunk modules optimization
3ms advanced chunk modules optimization
21ms module reviving
3ms module order optimization
7ms module id optimization
7ms chunk reviving
1ms chunk order optimization
15ms chunk id optimization
73ms hashing
2ms module assets processing
231ms chunk assets processing
477ms additional chunk assets processing
2ms recording
0ms additional asset processing
1ms chunk asset optimization
 94% asset optimization
[at-loader] Checking started in a separate process...

[at-loader] Ok, 0.478 sec.                                                                       850ms asset optimization

Now output with the plugin:

webpack: Compiling...
4167ms building modules
45ms sealing
2ms optimizing
0ms basic module optimization
12ms module optimization
1ms advanced module optimization
27ms basic chunk optimization
1ms chunk optimization
2ms advanced chunk optimization
0ms module and chunk tree optimization
0ms chunk modules optimization
3ms advanced chunk modules optimization
33ms module reviving
3ms module order optimization
12ms module id optimization
10ms chunk reviving
1ms chunk order optimization
21ms chunk id optimization
56ms hashing
2ms module assets processing
1876ms chunk assets processing
909ms additional chunk assets processing
1ms recording
1ms additional asset processing
0ms chunk asset optimization
 94% asset optimization
[at-loader] Checking started in a separate process...

[at-loader] Ok, 1.287 sec.
webpack: wait until bundle finished: /api/profile-info
webpack: wait until bundle finished: /i18n/en.json
MergetJsonsWebpackPlugin compilation started...
MergetJsonsWebpackPlugin compilation completed...                                                1493ms asset optimization
 95% emittingMergetJsonsWebpackPlugin emit starts...
MergetJsonsWebpackPlugin emit completed...                                                                 244ms emitting
Hash: 1e2541b49ad2db2b6351
Version: webpack 3.8.1
Time: 8792ms
                                       Asset     Size  Chunks                    Chunk Names
content/a0bb7ff631bd7ee48b810405f3005907.png   3.4 kB          [emitted]
                          app/main.bundle.js  4.43 MB       2  [emitted]  [big]  main
                      app/manifest.bundle.js  31.1 kB       5  [emitted]         manifest
                              ./i18n/en.json  28.7 kB          [emitted]
 + 20 hidden assets
[./node_modules/webpack/hot ^\.\/log$] (webpack)/hot nonrecursive ^\.\/log$ 170 bytes {3} [built]
       [] -> factory:0ms dependencies:0ms = 0ms
[./src/main/webapp/app lazy recursive] ./src/main/webapp/app lazy 160 bytes {2} [built]
       [] -> factory:0ms building:1ms = 1ms
[./src/main/webapp/app/app.main.ts] ./src/main/webapp/app/app.main.ts 588 bytes {2} [built]
       [] -> factory:0ms building:0ms = 0ms
[./src/main/webapp/app/app.module.ts] ./src/main/webapp/app/app.module.ts 3.38 kB {2} [built]
       [] -> factory:2ms building:1ms dependencies:1ms = 4ms
[./src/main/webapp/app/reports/index.ts] ./src/main/webapp/app/reports/index.ts 584 bytes {2} [built]
       [] -> factory:0ms building:0ms dependencies:5ms = 5ms
[./src/main/webapp/app/reports/prepare-reports/index-report.component.ts] ./src/main/webapp/app/reports/prepare-reports/index-report.component.ts 3.54 kB {2} [built]
       [] -> factory:1ms building:0ms dependencies:1ms = 2ms
[./src/main/webapp/app/reports/reports.module.ts] ./src/main/webapp/app/reports/reports.module.ts 1.82 kB {2} [built]
       [] -> factory:12ms building:351ms dependencies:2ms = 365ms
[./src/main/webapp/app/reports/reports.route.ts] ./src/main/webapp/app/reports/reports.route.ts 396 bytes {2} [built]
       [] -> factory:4ms building:16ms dependencies:0ms = 20ms
[./src/main/webapp/content/images/logo.png] ./src/main/webapp/content/images/logo.png 90 bytes {2} [built]
       [] -> factory:0ms building:33ms = 33ms
    + 1575 hidden modules
Child html-webpack-plugin for "index.html":
     2 assets
       1 module
webpack: Compiled successfully.
MergetJsonsWebpackPlugin emit starts...
MergetJsonsWebpackPlugin emit completed...
Hash: 1e2541b49ad2db2b6351
Version: webpack 3.8.1
Time: 9247ms
         Asset     Size  Chunks             Chunk Names
./i18n/es.json  30.2 kB          [emitted]
 + 23 hidden assets
[./node_modules/webpack/hot ^\.\/log$] (webpack)/hot nonrecursive ^\.\/log$ 170 bytes {3} [built]
       [] -> factory:0ms dependencies:0ms = 0ms
[./src/main/webapp/app lazy recursive] ./src/main/webapp/app lazy 160 bytes {2} [built]
       [] -> factory:0ms building:1ms = 1ms
[./src/main/webapp/app/app.main.ts] ./src/main/webapp/app/app.main.ts 588 bytes {2} [built]
       [] -> factory:0ms building:0ms = 0ms
[./src/main/webapp/app/app.module.ts] ./src/main/webapp/app/app.module.ts 3.38 kB {2} [built]
       [] -> factory:2ms building:1ms dependencies:1ms = 4ms
[./src/main/webapp/app/reports/index.ts] ./src/main/webapp/app/reports/index.ts 584 bytes {2} [built]
       [] -> factory:0ms building:0ms dependencies:5ms = 5ms
[./src/main/webapp/app/reports/prepare-reports/index-report.component.ts] ./src/main/webapp/app/reports/prepare-reports/index-report.component.ts 3.54 kB {2} [built]
       [] -> factory:1ms building:0ms dependencies:1ms = 2ms
[./src/main/webapp/app/reports/reports.module.ts] ./src/main/webapp/app/reports/reports.module.ts 1.82 kB {2} [built]
       [] -> factory:12ms building:351ms dependencies:2ms = 365ms
[./src/main/webapp/app/reports/reports.route.ts] ./src/main/webapp/app/reports/reports.route.ts 396 bytes {2} [built]
       [] -> factory:4ms building:16ms dependencies:0ms = 20ms
[./src/main/webapp/content/images/logo.png] ./src/main/webapp/content/images/logo.png 90 bytes {2} [built]
       [] -> factory:0ms building:33ms = 33ms
    + 1575 hidden modules
Child html-webpack-plugin for "index.html":
     2 assets
       1 module
webpack: Compiled successfully.

As you can see webpack: Compiled successfully. is emitted twice, so hot module reload is not functional when debugging in VSCode and I can see that the webapp is reloaded 2 or 3 times in the browser. I have created a very little project so you can test it yourself. You only need the frontend so it is not functional but you can run it with:

yarn start

And a a change in source to see the app reload twice. Then you can disable your plugin in webpack/webpack.dev.js to see the difference. (https://www.dropbox.com/s/k6gvwsd7dfgaqlv/test_4.6.1.zip?dl=0)

floralWallpaper commented 6 years ago

@jmesa-sistel - Did you ever resolve this? I believe I'm having a similar issue, which I logged here a couple days ago.

jmesa-sistel commented 6 years ago

@dgresko Nope, what I did is create another task in package.json to use this module, so manually I have to execute this new task to get my jsons merged when I modify a json. Not good but at least I can debug without problems.

floralWallpaper commented 6 years ago

@jmesa-sistel - thanks for your response. I think my issue is actually more related to this issue: https://github.com/tettusud/merge-jsons-webpack-plugin/issues/21 where I cannot have more than one object in the "groupBy" array. It did seem like webpack was compiling twice though, so I'm not sure. I've given up and am writing a node task instead. Thanks again.

tettusud commented 6 years ago

Hello all I am working on a di for this. I will release a new version once I test it.

tettusud commented 6 years ago

@dgresko I didn’t not understand the issue do you mean when you make modifications,that’s not automatically starting the compilation and merging the files?

floralWallpaper commented 6 years ago

No, apologies but my issue is here: https://github.com/tettusud/merge-jsons-webpack-plugin/issues/27. And I thought it may be related to this issue because it seemed to run ”compile” twice but I think it’s not related.

tettusud commented 6 years ago

@bridzius @jmesa-sistel @dgresko @fabgallo I have published new version 1.0.13 , that will resolve this issue.Thanks for your patience and support!