search

FormidableLabs / inspectpack

An inspection tool for Webpack frontend JavaScript bundles.
MIT License
591 stars 20 forks source link

BUG: Loaders (css-loader + style-loader) can generated different modules ending with identical relative paths (`./node_modules/PKG/FILE.css`) #125

Open boris-petrov opened 4 years ago

boris-petrov commented 4 years ago

Thanks for the great plugin!

This is my configuration:

new DuplicatesPlugin({
  emitError: true,
  verbose: true,
}),

This is the output I get:

WARNING in Missing sources: Expected 22, found 0.
Found map: {}

Duplicate Sources / Packages - Duplicates found! ⚠️

* Duplicates: Found 22 similar files across 22 code sources (both identical + similar)
  accounting for 219534 bundled bytes.
* Packages: Found 0 packages with 0 resolved, 0 installed, and 0 depended versions.

* Understanding the report: Need help with the details? See:
  https://github.com/FormidableLabs/inspectpack/#diagnosing-duplicates
* Fixing bundle duplicates: An introductory guide:
  https://github.com/FormidableLabs/inspectpack/#fixing-bundle-duplicates

I guess this is some bug that it doesn't show me what the duplicates are. Also, there are some missing sources but I don't know where these come from.

How do I debug this?

ryan-roemer commented 4 years ago

I'm guessing it is an issue of where you are running inspectpack from vs. the location of node_modules. For easier diagnosis, following these instructions to generate a stats file in the directory you normally run things: https://github.com/FormidableLabs/inspectpack/tree/0c2e5889c1d403c64c3b657577bde30b71ba7739#generating-a-stats-object-file

Then run the duplicates report on that file:

$ inspectpack -s /PATH/TO/stats.json -a duplicates -f text

You should receive information.

Then run the versions report:

$ inspectpack -s /PATH/TO/stats.json -a versions -f text

I'm guessing here is where things will report nothing.

If you can get me those outputs (feel free to sanitize as appropriate for your project if private), and the following information:

... and then with this info we can dig in more. Thanks!

boris-petrov commented 4 years ago

@ryan-roemer - thanks for the support and detailed write up.

Here are the results of running the two commands.

This is repeated 22 times for different CSS files:

inspectpack --action=duplicates
===============================

## Summary
* Extra Files (unique):         22
* Extra Sources (non-unique):   22
* Extra Bytes (non-unique):     219534

## `chunk.72eb4445de476d828ebb.js`
* c3/c3.css
  * Meta: Files 2, Sources 2, Bytes 4447
  0. (Files 1, Sources 1, Bytes 3829)
    (3829) /home/boris/project/node_modules/css-loader/dist/cjs.js!/home/boris/project/node_modules/c3/c3.css
  1. (Files 1, Sources 1, Bytes 618)
    (618) /home/boris/project/node_modules/style-loader/dist/cjs.js??ref--5-0!/home/boris/project/node_modules/css-loader/dist/cjs.js!/home/boris/project/node_modules/c3/c3.css

And the output for the -a versions:

inspectpack --action=versions
=============================

## Summary
* Packages with skews:      0
* Total resolved versions:  0
* Total installed packages: 0
* Total depended packages:  0
* Total bundled files:      0

Some background. This started happening recently and, seeing this output, I can tell what's going on - we migrated to using style-loader to dynamically import 3rd-party CSS files. We have 22 such files. Not sure why inspectpack is treating them as missing/duplicated/whatever.

Any ideas? Do you need me to do anything else?

P.S. Sorry, forgot to answer your other questions:

1) We're using Ember.js so I use Ember CLI with a plugin that uses webpack underneath and I've just added the DuplicatesPlugin plugin to its configuration. 2) node_modules is in the same directory I run Ember CLI in - the root of my project. I don't run inspectpack manually - that is done by webpack which is in turn run by an Ember CLI plugin.

ryan-roemer commented 4 years ago

Do all of the CSS files end with the string !/home/boris/project/node_modules/c3/c3.css?

And are they all actually uniquely different? (Maybe not because you have different loader (css-loader vs style-loader) processing them?)

There's are the underlying assumptions in inspectpack that:

  1. The last part of a ! concatenated webpack path string is the actual file. (This should be correct as it's a webpack internal).
    • Here, it looks like maybe all of these are /home/boris/project/node_modules/c3/c3.css or maybe duplicate pairs of the exact same file on disk.
  2. That the same relative file path after the last node_modules means a potential duplicate.

Here, to inspectpack it looks like webpack is repeatedly bundling the fie c3/c3.css which means duplicates. This also doesn't trigger for version skews because it's literally the same node_modules/c3 installation so there aren't multiple packages that could have skews.

I think I'm going to boil this down to future change for inspectpack to handle the same file as duplicates from a single package.

boris-petrov commented 4 years ago

Happy New Year!

Not sure what you mean by "do all CSS files end with that string". I have 22 different files that I include dynamically and there is a warning for each of them. Here is another example:

* video.js/dist/video-js.css
  * Meta: Files 2, Sources 2, Bytes 47924
  0. (Files 1, Sources 1, Bytes 47294)
    (47294) /home/boris/project/node_modules/css-loader/dist/cjs.js!/home/boris/project/node_modules/video.js/dist/video-js.css
  1. (Files 1, Sources 1, Bytes 630)
    (630) /home/boris/project/node_modules/style-loader/dist/cjs.js??ref--5-0!/home/boris/project/node_modules/css-loader/dist/cjs.js!/home/boris/project/node_modules/video.js/dist/video-js.css

As you see, the files after the last ! are always the same physical file on the drive. Doesn't that mean that they are identical? I'm not sure what css-loader and style-loader are doing exactly but the latter needs the former and doesn't work without it so I cannot try just with one.

I didn't understand in the end whether you think this is a bug in inspectpack? Do I really have duplicated files bundled?

ryan-roemer commented 4 years ago

Can I get a copy of both module chunks? You can find things a bit more easily probably with a webpack config of:

modules.exports = {
  mode: "development",
  devtool: false,
   // ...
  output: {
    // ...
    pathinfo: true
  }

... and then search on those full path strings in the comments in the bundle.

Or can I get a small minimal project that reproduces the issue with install and build instructions to produce a bundle that exhibits the same issue?

... if webpack has different, expected module sources for:

then there is an additional issue for inspectpack to handle as it violates either 1 or 2 of the current built-in assumptions inspectpack has for analysis that may be incorrect in these cases.

Thanks!

boris-petrov commented 4 years ago

Here is a reproduction project. It uses Ember CLI. If you're unfamiliar with that, just npm install first and then run ./node_modules/ember-cli/bin/ember server. You will see the warning at the bottom of the output. You can see and change webpack's configuration in ember-cli-build.js. Just restart the server after that.

Please tell me if you need anything more from me, I'll be glad to help.

ryan-roemer commented 4 years ago

Thanks for the repro project. With this diff:

diff --git a/ember-cli-build.js b/ember-cli-build.js
index 6dc7194..4c339fd 100644
--- a/ember-cli-build.js
+++ b/ember-cli-build.js
@@ -7,6 +7,11 @@ module.exports = function(defaults) {
   let app = new EmberApp(defaults, {
     autoImport: {
       webpack: {
+        mode: 'development',
+        devtool: false,
+        output: {
+          pathinfo: true
+        },
         module: {
           rules: [
             {

I'm able to get these snippets from dist/assets/chunk.52bfef93cfbd366411ed.js for c3/c3.css:

/***/ "./node_modules/c3/c3.css":
/*!********************************!*\
  !*** ./node_modules/c3/c3.css ***!
  \********************************/
/*! no static exports found */
/***/ (function(module, exports, __webpack_require__) {

var api = __webpack_require__(/*! ../style-loader/dist/runtime/injectStylesIntoStyleTag.js */ "./node_modules/style-loader/dist/runtime/injectStylesIntoStyleTag.js");
            var content = __webpack_require__(/*! !../css-loader/dist/cjs.js!./c3.css */ "./node_modules/css-loader/dist/cjs.js!./node_modules/c3/c3.css");

            content = content.__esModule ? content.default : content;

            if (typeof content === 'string') {
              content = [[module.i, content, '']];
            }

var options = {};

options.insert = function (element) { document.getElementsByTagName('head')[0].prepend(element); };
options.singleton = false;

var update = api(module.i, content, options);

var exported = content.locals ? content.locals : {};

module.exports = exported;

/***/ }),

/***/ "./node_modules/css-loader/dist/cjs.js!./node_modules/c3/c3.css":
/*!**********************************************************************!*\
  !*** ./node_modules/css-loader/dist/cjs.js!./node_modules/c3/c3.css ***!
  \**********************************************************************/
/*! no static exports found */
/***/ (function(module, exports, __webpack_require__) {

// Imports
var ___CSS_LOADER_API_IMPORT___ = __webpack_require__(/*! ../css-loader/dist/runtime/api.js */ "./node_modules/css-loader/dist/runtime/api.js");
exports = ___CSS_LOADER_API_IMPORT___(false);
// Module
exports.push([module.i, "/*-- Chart --*/\n.c3 svg {\n  font: 10px sans-serif;\n  -webkit-tap-highlight-color: rgba(0, 0, 0, 0);\n}\n\n.c3 path, .c3 line {\n  fill: none;\n  stroke: #000;\n}\n\n.c3 text {\n  -webkit-user-select: none;\n  -moz-user-select: none;\n  user-select: none;\n}\n\n.c3-legend-item-tile,\n.c3-xgrid-focus,\n.c3-ygrid,\n.c3-event-rect,\n.c3-bars path {\n  shape-rendering: crispEdges;\n}\n\n.c3-chart-arc path {\n  stroke: #fff;\n}\n\n.c3-chart-arc rect {\n  stroke: white;\n  stroke-width: 1;\n}\n\n.c3-chart-arc text {\n  fill: #fff;\n  font-size: 13px;\n}\n\n/*-- Axis --*/\n/*-- Grid --*/\n.c3-grid line {\n  stroke: #aaa;\n}\n\n.c3-grid text {\n  fill: #aaa;\n}\n\n.c3-xgrid, .c3-ygrid {\n  stroke-dasharray: 3 3;\n}\n\n/*-- Text on Chart --*/\n.c3-text.c3-empty {\n  fill: #808080;\n  font-size: 2em;\n}\n\n/*-- Line --*/\n.c3-line {\n  stroke-width: 1px;\n}\n\n/*-- Point --*/\n.c3-circle {\n  fill: currentColor;\n}\n\n.c3-circle._expanded_ {\n  stroke-width: 1px;\n  stroke: white;\n}\n\n.c3-selected-circle {\n  fill: white;\n  stroke-width: 2px;\n}\n\n/*-- Bar --*/\n.c3-bar {\n  stroke-width: 0;\n}\n\n.c3-bar._expanded_ {\n  fill-opacity: 1;\n  fill-opacity: 0.75;\n}\n\n/*-- Focus --*/\n.c3-target.c3-focused {\n  opacity: 1;\n}\n\n.c3-target.c3-focused path.c3-line, .c3-target.c3-focused path.c3-step {\n  stroke-width: 2px;\n}\n\n.c3-target.c3-defocused {\n  opacity: 0.3 !important;\n}\n\n/*-- Region --*/\n.c3-region {\n  fill: steelblue;\n  fill-opacity: 0.1;\n}\n.c3-region text {\n  fill-opacity: 1;\n}\n\n/*-- Brush --*/\n.c3-brush .extent {\n  fill-opacity: 0.1;\n}\n\n/*-- Select - Drag --*/\n/*-- Legend --*/\n.c3-legend-item {\n  font-size: 12px;\n}\n\n.c3-legend-item-hidden {\n  opacity: 0.15;\n}\n\n.c3-legend-background {\n  opacity: 0.75;\n  fill: white;\n  stroke: lightgray;\n  stroke-width: 1;\n}\n\n/*-- Title --*/\n.c3-title {\n  font: 14px sans-serif;\n}\n\n/*-- Tooltip --*/\n.c3-tooltip-container {\n  z-index: 10;\n}\n\n.c3-tooltip {\n  border-collapse: collapse;\n  border-spacing: 0;\n  background-color: #fff;\n  empty-cells: show;\n  -webkit-box-shadow: 7px 7px 12px -9px #777777;\n  -moz-box-shadow: 7px 7px 12px -9px #777777;\n  box-shadow: 7px 7px 12px -9px #777777;\n  opacity: 0.9;\n}\n\n.c3-tooltip tr {\n  border: 1px solid #CCC;\n}\n\n.c3-tooltip th {\n  background-color: #aaa;\n  font-size: 14px;\n  padding: 2px 5px;\n  text-align: left;\n  color: #FFF;\n}\n\n.c3-tooltip td {\n  font-size: 13px;\n  padding: 3px 6px;\n  background-color: #fff;\n  border-left: 1px dotted #999;\n}\n\n.c3-tooltip td > span {\n  display: inline-block;\n  width: 10px;\n  height: 10px;\n  margin-right: 6px;\n}\n\n.c3-tooltip .value {\n  text-align: right;\n}\n\n/*-- Area --*/\n.c3-area {\n  stroke-width: 0;\n  opacity: 0.2;\n}\n\n/*-- Arc --*/\n.c3-chart-arcs-title {\n  dominant-baseline: middle;\n  font-size: 1.3em;\n}\n\n.c3-chart-arcs .c3-chart-arcs-background {\n  fill: #e0e0e0;\n  stroke: #FFF;\n}\n\n.c3-chart-arcs .c3-chart-arcs-gauge-unit {\n  fill: #000;\n  font-size: 16px;\n}\n\n.c3-chart-arcs .c3-chart-arcs-gauge-max {\n  fill: #777;\n}\n\n.c3-chart-arcs .c3-chart-arcs-gauge-min {\n  fill: #777;\n}\n\n.c3-chart-arc .c3-gauge-value {\n  fill: #000;\n  /*  font-size: 28px !important;*/\n}\n\n.c3-chart-arc.c3-target g path {\n  opacity: 1;\n}\n\n.c3-chart-arc.c3-target.c3-focused g path {\n  opacity: 1;\n}\n\n/*-- Zoom --*/\n.c3-drag-zoom.enabled {\n  pointer-events: all !important;\n  visibility: visible;\n}\n\n.c3-drag-zoom.disabled {\n  pointer-events: none !important;\n  visibility: hidden;\n}\n\n.c3-drag-zoom .extent {\n  fill-opacity: 0.1;\n}\n", ""]);
// Exports
module.exports = exports;

/***/ }),

And so it looks like what get's generated by the plugins are:

Since both of the things end with the same path name (the original pure CSS file c3/c3.css) I will think about how best to handle this situation generally from inspectpack when two identical relative path names are legitimately different files (from plugin generation).