search

peerigon / extract-loader

webpack loader to extract HTML and CSS from the bundle
The Unlicense
317 stars 74 forks source link

SyntaxError: Unexpected token export #67

Closed carsonreinke closed 4 years ago

carsonreinke commented 4 years ago

Using the example with Node v8.15.0 produces the error SyntaxError: Unexpected token export at node_modules/extract-loader/lib/extractLoader.js:81:28.

The line 81 is trying to parse the source:

module.exports = "<!DOCTYPE html>\n<html>\n    <head>\n        <link rel=\"style\" href=\"" + require("./index.css") + "\">\n    </head>\n    <body>\n              <img src=\"" + require("./images/desk.svg") + "\">\n        <script src=\"index.js\"></script>\n    </body>\n</html>";
export default __webpack_public_path__ + "c58ff9d72352b98461e9b50bc7a28553.svg";
exports = module.exports = require("../node_modules/css-loader/dist/runtime/api.js")(true);
// Module
exports.push([module.id, "body {\n    font-size: 1em;\n}", "",{"version":3,"sources":["index.css"],"names":[],"mappings":"AAAA;IACI,cAAc;AAClB","file":"index.css","sourcesContent":["body {\n    font-size: 1em;\n}"]}]);

export default __webpack_public_path__ + "01f1344ea069d644fc25b1b869447810.css";

Here is the full stack:

Module build failed (from ./node_modules/extract-loader/lib/extractLoader.js):
[snip]/src/images/desk.svg:1
export default __webpack_public_path__ + "c58ff9d72352b98461e9b50bc7a28553.svg";
^^^^^^

SyntaxError: Unexpected token export
    at [snip]/node_modules/extract-loader/lib/extractLoader.js:81:28
    at Generator.next (<anonymous>)
    at step ([snip]/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at [snip]/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
    at new Promise (<anonymous>)
    at new F ([snip]/node_modules/core-js/library/modules/_export.js:36:28)
    at [snip]/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12
    at evalModule ([snip]/node_modules/extract-loader/lib/extractLoader.js:150:63)
    at [snip]/node_modules/extract-loader/lib/extractLoader.js:138:28
    at Generator.next (<anonymous>)
    at step ([snip]/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at [snip]/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)
 @ multi ./src/index.js ./src/index.html main[1]
stassribnyi commented 4 years ago

I believe it's related to file-loader also.

I've encountered the same issue with file-loader of version 5.2.0, so I rolled back to version 4.3.0 and it works like a charm, still didn't try [5.0.0, 5.1.0], so not sure which version starts causing a problem.

carsonreinke commented 4 years ago

@stassribnyi correct sir, downgrading fixed the issue. When I look at the source being provided, it is ES5 instead of ES6. I believe an adjustment to this project could help too.

module.exports = "<!DOCTYPE html>\n<html>\n    <head>\n        <link rel=\"style\" href=\"" + require("./index.css") + "\">\n    </head>\n    <body>\n        <img src=\"" + require("./images/desk.svg") + "\">\n        <script src=\"index.js\"></script>\n    </body>\n</html>";
module.exports = __webpack_public_path__ + "c58ff9d72352b98461e9b50bc7a28553.svg";
exports = module.exports = require("../node_modules/css-loader/dist/runtime/api.js")(true);
// Module
exports.push([module.id, "body {\n    font-size: 1em;\n}", "",{"version":3,"sources":["index.css"],"names":[],"mappings":"AAAA;IACI,cAAc;AAClB","file":"index.css","sourcesContent":["body {\n    font-size: 1em;\n}"]}]);

module.exports = __webpack_public_path__ + "01f1344ea069d644fc25b1b869447810.css";
carsonreinke commented 4 years ago

Looking at solving this. The issue is simple VM (just like CLI) for Node will not interpret ES6 import/export statements. Copying and pasting the code ES6 code above will produce an error on any Node CLI versions.

One solution would to have Babel transform this, since there is already an odd dependency on babel-runtime 6.

Another would be to write the script to file and run it and extract the data. I like something like this better because it would potentially remove a dependency and locking people on Babel 6.

cybercase commented 4 years ago

I bumped into this issue too. Looking at the file-loader changelog for v 5.0.0 here I read they've switched to ES modules by default. To restore the pre-version-5 behaviour now we need to explicitly disable the ES modules, by setting esModule: false into the file-loader's options. I just wonder how it's working fine for them

carsonreinke commented 4 years ago

@cybercase yes, looks like that is an option https://github.com/webpack-contrib/file-loader#esmodule.

LZQCN commented 4 years ago 
content.replace(/export default/g, "module.exports =")

Maybe it's useful to insert this code in the right place. The question is where to insert it?

carsonreinke commented 4 years ago

@LZQCN are you suggesting instead of Babel just using this regex? Is it that simple?

cybercase commented 4 years ago

@carsonreinke To deal with the new file-loader version, yes, it would be that simple. However, there's a past PR ( #26 ) where the maintainer already discussed a fix of this kind, and expressed the willing to use babel for proper transpilation, so I've opened a PR ( #69 ) to handle es modules using babel. I'm waiting for feedback from @jhnns

pbowyer commented 4 years ago

After too many hours trying to work out why my Webpack config kept failing, I am very glad to have found this issue.

ERROR in ./src/test.html
Module build failed (from ./node_modules/extract-loader/lib/extractLoader.js):
D:\projects\src\images\london.jpg:1
export default __webpack_public_path__ + "dda88ae5192fe238f8384d1c0a2143db.jpg";
^^^^^^

SyntaxError: Unexpected token export
    at new Script (vm.js:79:7)
    at D:\projects\node_modules\extract-loader\lib\extractLoader.js:81:28
    at Generator.next (<anonymous>)
    at step (D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
    at D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:35:14
    at new Promise (<anonymous>)
    at new F (D:\projects\node_modules\babel-runtime\node_modules\core-js\library\modules\_export.js:36:28)
    at D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:14:12
    at evalModule (D:\projects\node_modules\extract-loader\lib\extractLoader.js:150:63)
    at D:\projects\node_modules\extract-loader\lib\extractLoader.js:138:28
    at Generator.next (<anonymous>)
    at step (D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
    at D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ./src/js/app.js 8:0-23
error Command failed with exit code 2.

Rolling back to file-loader@4.3 fixed the problem for me

jhnns commented 4 years ago

Thanks for reporting ❤️. We're working on this (and other) issues.

rodrigoyoshida commented 4 years ago

Same here, unfortunately the esModule: false option did not work for me. Rolling back to file-loader@4.3 fixed the problem for me as well.

cybercase commented 4 years ago

@rodrigoyoshida I don't know the exact version of the file-loader you were using before, but I've found that they fixed a typo in v. 5.0.1 about the name of esModule option. https://github.com/webpack-contrib/file-loader/pull/346 Maybe updating to the latest version could make the esModule: false option work?

carsonreinke commented 4 years ago

@cybercase @rodrigoyoshida looks like definitely v5.0.2 has it fixed https://github.com/webpack-contrib/file-loader/blob/v5.0.2/src/index.js#L63

thequailman commented 4 years ago

Still seeing this with 5.0.2, I think:

ERROR in   Error: Child compilation failed:
  Module build failed (from ./node_modules/extract-loader/lib/extractLoader.js):
  /home/thequailman/Code/app/ui/src/images/icon-192.png:1
  export default __webpack_public_path__ + "ui/icon-192.c017b0bc34d9b6470c9020563ba18fdb.png";
  ^^^^^^
  SyntaxError: Unexpected token export

  - extractLoader.js:98 
    [ui]/[extract-loader]/lib/extractLoader.js:98:28

  - Generator.next

  - asyncToGenerator.js:17 step
    [ui]/[babel-runtime]/helpers/asyncToGenerator.js:17:30

  - asyncToGenerator.js:35 
    [ui]/[babel-runtime]/helpers/asyncToGenerator.js:35:14

  - new Promise

  - _export.js:36 new F
    [ui]/[core-js]/library/modules/_export.js:36:28
kevincox commented 4 years ago

Are there any plans to support es modules in the extract-loader?

carsonreinke commented 4 years ago

@thequailman I was able to get it to work and created this example: https://github.com/carsonreinke/extract-loader-67

@kevincox yes, @cybercase was kind enough to put in a PR #69

kamleshlikhare commented 4 years ago

/// esModule: false, will solve your issue { loader: "file-loader", options: { name: "images/[name]-[hash:8].[ext]", esModule: false },

        }
carsonreinke commented 4 years ago

@kamleshlikhare you don't need all the name stuff, just enabling option esModule.

Wondering if there is a way to enable that globally...

scott-ln commented 4 years ago

The esModule: false solution isn't working for me, although I'm not very expert at package management and not sure if file-loader@5.0.2 is actually being used when I run my build, since older versions are present in the tree.

scott@laptop MINGW64 ~/project (master)
$ npm ls file-loader
project@1.0.0 C:\Users\scott\project
+-- UNMET PEER DEPENDENCY @storybook/react@5.3.14
| `-- @storybook/core@5.3.14
|   `-- file-loader@4.3.0
+-- base64-inline-loader@1.1.1
| `-- file-loader@1.1.11
+-- file-loader@5.0.2
`-- react-scripts@3.4.0
  `-- file-loader@4.3.0

npm ERR! peer dep missing: @storybook/react@^3, required by storybook-addon-react-live-edit@2.0.4

scott@laptop MINGW64 ~/project (master)
$ npm ls extract-loader
project@1.0.0 C:\Users\scott\project
`-- extract-loader@4.0.3
carsonreinke commented 4 years ago

@scott-ln well, you can't use 4.0.3, so you probably need to tweak those dependencies to get it up to >=5.0.2. Try setting file-loader dependency to =5.0.2 and see what happens.

Luiz-Monad commented 4 years ago

Isn't better doing what the val-loader does ?

import Module from 'module';
function exec(code, loaderContext) {
  const { resource, context } = loaderContext;

  const module = new Module(resource, parentModule);

  // eslint-disable-next-line no-underscore-dangle
  module.paths = Module._nodeModulePaths(context);
  module.filename = resource;

  // eslint-disable-next-line no-underscore-dangle
  module._compile(code, resource);

  return module.exports;
}

https://github.com/webpack-contrib/val-loader/blob/master/src/index.js#L10-L23

Or the Module would suffer from the same problem?

Luiz-Monad commented 4 years ago

Perhaps using ChildCompilation?

more...

After reading the WebPack to understand what happens after the File-Loader. The file-loader is called by the Load-Runner from the NormalModule when the build happens. After it the Acorn Parser runs and produces AST for the Harmony. So we are at loss, there's no way to make webpack do that for us, because all `export default` is stripped in the AST level when WebPack convert the output to its Harmony dependencies. The load-runner uses `require` and rely on loaders being CJS (even thou they output ES modules). There's no way around except for creating a child compilation, or using babel. (as the Terser won't remove `exports default` clause, at least it appears to do so) I think the cleanest way would be creating a child compilation targeting Node so WebPack can deal with it for us. The easiest + also correct way would be babel, but incurs babel dependency. sources: https://github.com/webpack/loader-runner/blob/160c809d8d03cebe282fdb3b9d3cecedd4f6da5b/lib/loadLoader.js#L5 https://github.com/webpack/webpack/blob/503e6561d583f143896151a5b22e771f554773ec/lib/NormalModule.js#L793 https://github.com/webpack/webpack/blob/503e6561d583f143896151a5b22e771f554773ec/lib/javascript/JavascriptParser.js#L1602 https://github.com/webpack/webpack/blob/a488d073c3a0014ddafe5a56d34b57a7c934f94f/lib/dependencies/HarmonyExportDependencyParserPlugin.js#L125

carsonreinke commented 4 years ago

@Luiz-Monad I think you might be reading into a bit much, the problem is simple, vm.Script does not support import syntax, just like the Node REPL. It is used here: https://github.com/peerigon/extract-loader/blob/master/src/extractLoader.js#L98

Will that Module work? MAYBE! I have not heard of it before, but it could be a better option then Babel.

Luiz-Monad commented 4 years ago

Actually I was over-complicating things.

I discovered this file. https://github.com/jantimon/html-webpack-plugin/blob/master/lib/compiler.js

I just took this file and turned it into a loader, it does exactly what I needed. I need it to process exactly like webpack would do it, and extract the resources (and preserve ES for tree-shaking reasions), that's why I was thinking in a Child Compilation, but it would be doing twice if I used this module and also child-compilation. It does indeed solve the problem with modules by using Webpack itself. And it actually does extract the resources because the require runs inside the loader-loaders ! and Webpack already taps require.

I just plugged it after my html-loader, works like a charm.

simonppg commented 4 years ago

After too many hours trying to work out why my Webpack config kept failing, I am very glad to have found this issue.

ERROR in ./src/test.html
Module build failed (from ./node_modules/extract-loader/lib/extractLoader.js):
D:\projects\src\images\london.jpg:1
export default __webpack_public_path__ + "dda88ae5192fe238f8384d1c0a2143db.jpg";
^^^^^^

SyntaxError: Unexpected token export
    at new Script (vm.js:79:7)
    at D:\projects\node_modules\extract-loader\lib\extractLoader.js:81:28
    at Generator.next (<anonymous>)
    at step (D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
    at D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:35:14
    at new Promise (<anonymous>)
    at new F (D:\projects\node_modules\babel-runtime\node_modules\core-js\library\modules\_export.js:36:28)
    at D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:14:12
    at evalModule (D:\projects\node_modules\extract-loader\lib\extractLoader.js:150:63)
    at D:\projects\node_modules\extract-loader\lib\extractLoader.js:138:28
    at Generator.next (<anonymous>)
    at step (D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
    at D:\projects\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ./src/js/app.js 8:0-23
error Command failed with exit code 2.

Rolling back to file-loader@4.3 fixed the problem for me

Rolling back works for me, thanks

scott-ln commented 4 years ago

@carsonreinke wrote:

@scott-ln well, you can't use 4.0.3, so you probably need to tweak those dependencies to get it up to >=5.0.2. Try setting file-loader dependency to =5.0.2 and see what happens.

Sorry, I'm not sure what you mean? 4.0.3 is the latest version of extract-loader, not file-loader (which is installed at 5.0.2 in this scenario).

carsonreinke commented 4 years ago

@scott-ln :man_facepalming: , sorry

@Luiz-Monad it sounds like this might be a better solution to using Babel, do you have anything you can share and we could possible get a PR together?

jhnns commented 4 years ago

The issue should be solved with v5. Please re-open if that's not the case.

mnpenner commented 4 years ago

@Luiz-Monad Can you elaborate a bit? It sounds like you're trying to do exactly what I'm trying to do.

I was trying to harvest some code from val-loader but even after I hacked around the "Unexpected token export" its choking on the next thing because webpack hasn't finished processing my file.

i.e., I want the evaluated/executed version of a file.

How did you convert compiler.js into a loader?