tj / consolidate.js

Template engine consolidation library for node.js
3.48k stars 353 forks source link

Webpack build fails #295

Closed AmitMY closed 7 years ago

AmitMY commented 7 years ago

I am trying to build my server app with webpack, so I don't have to have npm on my server (I can't have it).

consolidate throws many errors on unresolved modules, unless I install them manually, and if I do npm i again it is back to "nothing is installed".

I am getting the following, but for many many more packages:

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'mote' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 674:49-64 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'nunjucks' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 762:57-76 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'ractive' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 736:55-73 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'swig' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 244:49-64 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'templayed' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 468:59-79 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'then-jade' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 163:31-51 179:31-51 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'toffee' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 688:53-70 697:53-70 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'underscore' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 514:61-82 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'walrus' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 587:53-70 @ ./node_modules/consolidate/index.js @ ./index.js

ERROR in ./node_modules/consolidate/lib/consolidate.js Module not found: Error: Can't resolve 'whiskers' in 'C:\Users\user\Desktop\Projects\instance-manager\server\node_modules\consolidate\lib' @ ./node_modules/consolidate/lib/consolidate.js 400:57-76 409:57-76 @ ./node_modules/consolidate/index.js @ ./index.js

I believe that is because you use devDependencies instead of dependencies for some of your actual dependencies.

honpery commented 7 years ago

@AmitMY Do you solve this problem?

AmitMY commented 7 years ago

Nope :(

avevlad commented 7 years ago

+1

doowb commented 7 years ago

consolidate is designed this way. The template engines are only required in when they are needed. That's why the documentation says that you install consolidate and the engine you want to use:

NOTE: you must still install the engines you wish to use, add them to your package.json dependencies.

Since webpack is trying to statically analyze your code to determine which dependencies are used at build time (instead of runtime), it won't be able to find all of the engines that you don't want to use.

You can either install all of the engines and add them to your project's package.json or configure webpack to ignore the engines that you don't want to use and only install the engines that you want to use. I don't know the specifics of webpack configurations, so that's a good question for stack overflow or the webpack repository.

I'm going to close this because I think it would be a bad idea to add all of the engines as dependencies to consolidate.

If someone does find a good webpack configuration and wants to add it to the README, I think that would be useful and would accept that PR.

che-wf commented 6 years ago

This seems to be a poor user experience when the user's only options are

This should be taken care of within the package. Is there any fix to this in the works?

doowb commented 6 years ago

This seems to be a poor user experience

This poor user experience comes from how webpack handles this and not how consolidate handles it. Like mentioned above, consolidate should not include all template engines as dependencies since this will cause a lot of unnecessary libraries to be included. (Which would cause the bundled webpack package to be even bigger, which would cause more complaints.)

This should be taken care of within the package

I disagree. Every node module shouldn't have to change their code to accommodate edge cases created by package bundlers. The package bundler should fix this since this will fix the problem at the root and ensure that many more modules will work without minimal amounts of change.

Is there any fix to this in the works?

I think this should be a question for webpack. If I was doing this in webpack, I'd have something either built-in or as an option to not throw an error if modules being required are also found in the devDependencies. This should be possible even with static analysis.

If someone does find a good webpack configuration and wants to add it to the README, I think that would be useful and would accept that PR.

I'll still accept PRs. If there is a configuration that can be added to package.json that will work with webpack, I'd accept a PR for that too.

che-wf commented 6 years ago

...consolidate should not include all template engines as dependencies since this will cause a lot of unnecessary libraries to be included.

I concur. That would. My thought is that it would be better if Webpack or similar bundlers wouldn't throw errors if the packages were unnecessary.

Every node module shouldn't have to change their code to accommodate edge cases...

I hardly think of Webpack bundles as an edge case. It's a pretty standard use case. Consolidate is the first package I've worked with that has had this specific issue. So, I'm more inclined to think of it as a package issue than a bundler issue.

doowb commented 6 years ago

I don't mean webpack or other bundlers are edge cases. I mean the bundlers not handling how require is used here is an edge case for bundlers. It's something webpack doesn't expect but is still a valid use. I think this is valid because it's been the documented use of consolidate from the beginning.

I don't think modules should have to handle it because bundlers handle require statements in different ways and it would be difficult for modules to try to support all of the different ways. If a new bundler comes along that does something different, I don't think the modules should have to make changes.

From my experience with webpack complaining about how require is used, I don't see a good solution that can be implemented in consolidate. One idea I have is to create a function that would call require, then use that function instead of require directly. When doing this before, I've seen webpack throw errors saying that require is used in an unexpected way. This may be just as bad as the other errors. This would also cause webpack to not even try to bundle the necessary template engines. This would also be the case if we added a browser config to package.json to exclude the template engines.

che-wf commented 6 years ago

@doowb I see what you're saying now. Thanks for explaining that. I'm sure that will be useful for others who stumble on this post too.

homerjam commented 6 years ago

I agree that there should be a way to fix this in webpack and the root of the problem lies there. However I'm not sure a fix would be accepted as in the large majority of these cases you want an error to be thrown.

Unfortunately I think a solution will probably need to come from consolidate - perhaps this could be useful?

FWIW I'm using this to silence errors for now (remove the ones you actually use):

// webpack.config.js
...
plugins: [
  new webpack.IgnorePlugin(/(atpl|bracket|dot|doT.js|dust|dustjs-linkedin|eco|ect|ejs|haml|haml-coffee|hamlet|hiredis|handlebars|hogan|htmling|jade|jazz|jqtpl|just|liquor|lodash|marko|mote|mustache|nunjucks|plates|pug|QEJS|ractive|react|slm|swig|swig|teacup|templayed|twig|liquid|toffee|underscore|vash|walrus|whiskers)/),
]
...
Dawidpol commented 5 years ago

For me the plugin solution did not worked, I had to add this to externals and remove the one I use:

  externals: [
    'atpl',
    'bracket',
    'dot',
    'doT.js',
    'dust',
    'dustjs-linkedin',
    'eco',
    'ect',
    'ejs',
    'haml',
    'haml-coffee',
    'hamlet',
    'hiredis',
    'hogan',
    'htmling',
    'jade',
    'jazz',
    'jqtpl',
    'just',
    'liquor',
    'marko',
    'mote',
    'mustache',
    'nunjucks',
    'plates',
    'pug',
    'QEJS',
    'ractive',
    'react',
    'slm',
    'swig',
    'swig',
    'teacup',
    'templayed',
    'twig',
    'liquid',
    'toffee',
    'underscore',
    'vash',
    'walrus',
    'whiskers',
    'bracket-template',
    'hamljs',
    'hogan.js',
    'react-dom/server',
    'tinyliquid',
    'lodash',
    'handlebars'
  ],
Flamenco commented 5 years ago

@Dawidpol You left a comma out after 'tinyliquid'.

wald-tq commented 4 years ago

Another option is to specify consolidate as 'external' in webpack, with that you don't add it to the bundle and load it from node_modules.

webpack.config.js:

externals: {
  consolidate: 'commonjs consolidate'

see: https://webpack.js.org/configuration/externals/

homerjam commented 4 years ago

Back here again! Thanks @wald-tq - this is my preferred solution

Another option is to specify consolidate as 'external' in webpack, with that you don't add it to the bundle and load it from node_modules.

webpack.config.js:

externals: {
  consolidate: 'commonjs consolidate'

see: webpack.js.org/configuration/externals

jbcastro commented 3 years ago

I had this problem, it stemmed from Visual Studio Code. I was writing "const" but it took that as autocomplete or something as "cons" so it added "import cons from "consolidate";" to the top of my file. I finally figured it out by deleting the consolidate folder from node_modules, it then pointed to which component file was showing "import cons from "consolidate";" at the top.

So delete consolidate folder from node_modules, find what file is asking for "import cons from "consolidate";", delete that line. And if you're so inclined do NPM install again.