Module not found error can't resolve 'ws' in puppeteer-core/lib

kamira commented 6 years ago

Puppeteer version: 1.9.0
Platform / OS version: Oracle Linux 7
Node.js version: 8.11.4

What steps will reproduce the problem?

const puppeteer = require("puppeteer-core");
    (async () => {
      const browser = await puppeteer.launch({
        executablePath: '../../chrome/chrome'
      });
      const page = await browser.newPage();
      await page.goto(DashboardConstants.LANDING_PAGE_PATH + "&embed=true");
      await page.pdf({path: "example.pdf", fullPage: true, format: "A4"});

      await browser.close();
    })();
  }

git clone kibana
git checkout v6.4.2
add puppeteer-core in package.json
download chrome and extract at src/core_plugins/kibana/
edit src/core_plugins/kibana/public/dashboard/dashboard_app.js with code

What is the expected result?

I should Pass and kibana will be running

What happens instead?

I got Module not found error message.

Error Message:

error  [05:44:03.811] [fatal] Error: Optimizations failure.
8270 modules
ERROR in ./node_modules/puppeteer-core/lib/WebSocketTransport.js
Module not found: Error: Can't resolve 'ws' in '/home/ELK/kibana-test/kibana/node_modules/puppeteer-core/lib'

at FsOptimizer.failedStatsToError (/home/ELK/kibana-test/kibana/src/optimize/base_optimizer.js:429:17)
at compiler.run (/home/ELK/kibana-test/kibana/src/optimize/fs_optimizer.js:36:26)
at onCompiled (/home/ELK/kibana-test/kibana/node_modules/webpack/lib/Compiler.js:241:12)
at applyPluginsAsync.err (/home/ELK/kibana-test/kibana/node_modules/webpack/lib/Compiler.js:514:14)
at next (/home/ELK/kibana-test/kibana/node_modules/tapable/lib/Tapable.js:202:11)
at Compiler.<anonymous> (/home/ELK/kibana-test/kibana/node_modules/webpack/lib/CachePlugin.js:78:5)
at Compiler.applyPluginsAsyncSeries (/home/ELK/kibana-test/kibana/node_modules/tapable/lib/Tapable.js:206:13)
at compilation.seal.err (/home/ELK/kibana-test/kibana/node_modules/webpack/lib/Compiler.js:511:11)
at Compilation.applyPluginsAsyncSeries (/home/ELK/kibana-test/kibana/node_modules/tapable/lib/Tapable.js:195:46)
at self.applyPluginsAsync.err (/home/ELK/kibana-test/kibana/node_modules/webpack/lib/Compilation.js:671:19)
at Compilation.applyPluginsAsyncSeries (/home/ELK/kibana-test/kibana/node_modules/tapable/lib/Tapable.js:195:46)
at self.applyPluginsAsync.err (/home/ELK/kibana-test/kibana/node_modules/webpack/lib/Compilation.js:662:11)
at next (/home/ELK/kibana-test/kibana/node_modules/tapable/lib/Tapable.js:202:11)
at /home/ELK/kibana-test/kibana/node_modules/uglifyjs-webpack-plugin/dist/index.js:282:11
at step (/home/ELK/kibana-test/kibana/node_modules/uglifyjs-webpack-plugin/dist/uglify/index.js:90:11)
at done (/home/ELK/kibana-test/kibana/node_modules/uglifyjs-webpack-plugin/dist/uglify/index.js:99:22)
at /home/ELK/kibana-test/kibana/node_modules/uglifyjs-webpack-plugin/dist/uglify/index.js:105:15
at /home/ELK/kibana-test/kibana/node_modules/worker-farm/lib/farm.js:191:19
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)

JoelEinbinder commented 6 years ago

Kibana looks like they have quite an extensive dependency system on top of just package.json. I'd file the issue there. puppeteer-core definitely depends on ws and it should be installed when you add it.

AirborneEagle commented 5 years ago

I am having the same issue, But I am not sure I understand what the fix was? I ran this, but it did not change anything. npm install --save ws

What was done to solve the OP problem?

aslushnikov commented 5 years ago

@AirborneEagle this seems to be a downstream issue, please file a bug to kibana

AirborneEagle commented 5 years ago

I am confused. I am not using Kibana. I am using puppeteer in my own application. More specifically I am using pdf-puppeteer. https://github.com/westmonroe/pdf-puppeteer#readme

AnthoniG commented 5 years ago

Did this ever get resolved? I am getting the same error, either if I use puppeteer or puppeteer-core I am using inside Electron if that makes any difference but it shouldn't.

Help ?

tomermes commented 5 years ago

Did this ever get resolved? I am getting the same error, either if I use puppeteer or puppeteer-core I am using inside Electron if that makes any difference but it shouldn't.

Help ?

It happened to me when I accidentally tried to ran puppeteer from inside a browser code. If that's your case by any chance.

AnthoniG commented 5 years ago

@tomermes Yea it's similar. Electron acts like a browser in many ways, because it uses Chromium etc under the hood.

Tried looking around and it seems that you can not use Puppeteer inside Electron :disappointed:

aslushnikov commented 5 years ago

Tried looking around and it seems that you can not use Puppeteer inside Electron

@AnthoniG Electron has a lot of powerful APIs on its own. Why would you want to?

rmondesilva commented 5 years ago

I am getting the same error while compiling.

ERROR in ./node_modules/puppeteer/lib/WebSocketTransport.js
Module not found: Error: Can't resolve 'ws' in 'c:\...\project\node_modules\puppeteer\lib'

How to fix this? I checked the path, it exist.

AnthoniG commented 5 years ago

@aslushnikov Electron is very powerful yes. However in order to automate scraping simple websites I have to use IPC and it gets very messy very quickly.

Using puppeteer sandbox I could do it in a few less lines and there was no messing about with IPC at all. So having this inside Electron, or if there's anything similar you might now, would be absolutely great.

kalimuthu123 commented 5 years ago

I am getting the same error while compiling.
ERROR in ./node_modules/puppeteer/lib/WebSocketTransport.js
Module not found: Error: Can't resolve 'ws' in 'c:\...\project\node_modules\puppeteer\lib'
How to fix this? I checked the path, it exist.

I also tried the same bug when i used to webpack it does not work for me

stychu commented 5 years ago

Having the same trouble. I want to run puppeteer in the context of the electron. So when I run script with pupeeter scraping like node test.js its working. But when I require it inside the electron app to be able to run this script on button click from within electron app I get

./node_modules/puppeteer-core/lib/WebSocketTransport.js
Module not found: Can't resolve 'ws' in .....

AmmarHalees commented 5 years ago

Guys, if you're attempting to use it from the create-react-app bundle, then it's not going to work. According to their npm page:

https://www.npmjs.com/package/ws

Note:

This module does not work in the browser. The client in the docs is a reference to a back end with the role of a client in the WebSocket communication. Browser clients must use the native WebSocket object. To make the same code work seamlessly on Node.js and the browser, you can use one of the many wrappers available on npm, like isomorphic-ws.

aslushnikov commented 5 years ago

Also, for bundling Puppeteer for browser use, see https://github.com/GoogleChrome/puppeteer/blob/master/utils/browser/README.md

dfoverdx commented 5 years ago

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.

const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};

Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

9oelM commented 5 years ago

@dfoverdx +1 for your solution. I was using webpack too (it's not a web application tho, it's just running on node) and it definitely worked for me.

My initial error output was as follows:

$ webpack && node dist/bundle.js
Hash: 1b2fd47f36807b1b273d
Version: webpack 4.38.0
Time: 1573ms
Built at: 07/28/2019 10:10:40 PM
    Asset      Size  Chunks             Chunk Names
bundle.js  1.53 MiB    main  [emitted]  main
Entrypoint main = bundle.js
[0] fs (ignored) 15 bytes {main} [built]
[1] ./BrowserFetcher (ignored) 15 bytes {main} [built]
[2] util (ignored) 15 bytes {main} [built]
[3] util (ignored) 15 bytes {main} [built]
[4] rimraf (ignored) 15 bytes {main} [built]
[5] child_process (ignored) 15 bytes {main} [built]
[6] readline (ignored) 15 bytes {main} [built]
[7] ./node6/lib/Puppeteer (ignored) 15 bytes {main} [built]
[./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 472 bytes {main} [built]
[./node_modules/webpack/buildin/module.js] (webpack)/buildin/module.js 497 bytes {main} [built]
[./src/index.ts] 3.25 KiB {main} [built]
    + 78 hidden modules

ERROR in ./node_modules/puppeteer/lib/WebSocketTransport.js
Module not found: Error: Can't resolve 'ws' in '/Users/jm/Documents/Joel/scraper/node_modules/puppeteer/lib'
 @ ./node_modules/puppeteer/lib/WebSocketTransport.js 16:18-31
 @ ./node_modules/puppeteer/lib/Launcher.js
 @ ./node_modules/puppeteer/lib/Puppeteer.js
 @ ./node_modules/puppeteer/index.js
 @ ./src/index.ts
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

But when I just added

externals: {
        puppeteer: 'require("puppeteer")',
        // ...
},

to webpack config, it worked like a charm:

$ webpack && node dist/bundle.js
Hash: 1bd4c72ee57858351a97
Version: webpack 4.38.0
Time: 1117ms
Built at: 07/28/2019 10:26:41 PM
    Asset      Size  Chunks             Chunk Names
bundle.js  17.7 KiB    main  [emitted]  main
Entrypoint main = bundle.js
[./src/index.ts] 3.25 KiB {main} [built]
[puppeteer] external "require(\"puppeteer\")" 42 bytes {main} [built]
✨  Done in 5.34s.

shubuu97 commented 5 years ago

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

Can anyone tell me where to add this config?

9oelM commented 5 years ago

@shubuu97 Inside webpack.config.js. Here's my config as an example:


const path = require('path');

module.exports = {
  entry: './src/index.ts',
  devtool: 'inline-source-map',
  mode: 'development',
  // the webpack config just works
  // SEE https://github.com/webpack/webpack/issues/1599
  target: 'node',
  node: {
    __dirname: false,
    __filename: false,
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
      {
        test: /\.ts$/,
        loader: 'tslint-loader',
        enforce: 'pre',
        options: {
          fix: true,
          tsConfigFile: 'tsconfig.json',
        },
      },
    ],
  },
  resolve: {
    extensions: ['.tsx', '.ts', '.js'],
  },
  output: {
    filename: 'bundle.js',
    path: path.resolve(__dirname, 'dist'),
  },
  externals: {
    puppeteer: 'require("puppeteer")',
    fs: 'require("fs")',
  },
};

shubuu97 commented 5 years ago

Do we need to do anything else after adding this because i'm still facing the same error

kalimuthu123 commented 5 years ago

i got an error we require is not defined

dfoverdx commented 5 years ago

i got an error we require is not defined

@kalimuthu123 what environment are you running it in? My solution was for use in Node. Browsers would not have require defined.

I know I saw documentation once for running it in the browser, but I can't seem to find it now. Since you don't need to download Chromium for use in the browser, you should use puppeteer-core.

Nisthar commented 5 years ago

@dfoverdx I followed the guide for building puppeteer-web.js

I have this code inside background.js file

const puppeteer = require('puppeteer');
console.log(puppeteer);

I am getting

Uncaught ReferenceError: puppeteer is not defined

PwntasticKev commented 4 years ago

Any resolution to this? Ive tried all these solutions but still get the Can't resolve 'ws'

kamira commented 4 years ago

@dfoverdx I followed the guide for building puppeteer-web.js

I have this code inside background.js file
const puppeteer = require('puppeteer');
console.log(puppeteer);
I am getting
Uncaught ReferenceError: puppeteer is not defined

Do you install puppeteer first? It is ok which I test from Repl.it .

frey-1024 commented 4 years ago

这个是webpack include 的问题，可以把test文件写到src外面，并修该package.js 的jest配置项即可

ohmygodvt95 commented 4 years ago

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

+1 for your solution save my life

xywc-s commented 4 years ago

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

it's helpful for me. Thanks for u.

bfp4 commented 4 years ago

how do i fix this problem if Im using react-native?

zoffyzhang commented 4 years ago

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

thx, you just save my ass

afaher commented 4 years ago

Where the hell is my fucking best friend

On Mon, Nov 9, 2020 at 1:03 AM Zoffy Zhang notifications@github.com wrote:

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.

const config = { // ... externals: { puppeteer: 'require("puppeteer")', // ... },};

Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

thx, you just save my ass

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/puppeteer/puppeteer/issues/3466#issuecomment-723806667, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARGBR2JXFKRDXIJMDR4LUN3SO6H5DANCNFSM4GAE743Q .

lint-fengfeng commented 3 years ago

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do.
Can anyone tell me where to add this config?

For me, this happened when bundling with webpack, intending the bundle to be used in node. The trick was to set puppeteer as an external.
const config = {
    // ...
    externals: {
        puppeteer: 'require("puppeteer")',
        // ...
    },
};
Wrapping it in require() is important because puppeteer doesn't have a global export like jQuery and React do. after using this config， web throw an error

RockNHawk commented 1 year ago

@tomermes Yea it's similar. Electron acts like a browser in many ways, because it uses Chromium etc under the hood.

Tried looking around and it seems that you can not use Puppeteer inside Electron 😞

Each Electron app has a single main process, which acts as the application's entry point. The main process runs in a Node. js environment, meaning it has the ability to require modules and use all of Node. js APIs.

If scripts run in electron main process, it's a nodejs environment not browser enivronment,can use fs API etc..., but why still thow this error ? (Note. call puppeter.launch() the chrome.exe can started, but throw error )

ws does not work in the browser. Browser clients must use the native WebSocket object

puppeteer / puppeteer

Module not found error can't resolve 'ws' in puppeteer-core/lib #3466