spinlud / linkedin-jobs-scraper

151 stars 41 forks source link

Protocol error (Target.setDiscoverTargets): Target closed while running in AWS EC2 instance #29

Closed aditya314 closed 2 years ago

aditya314 commented 2 years ago

Using this configuration along with LI_AT_COOKIE but getting this error. It works completely fine on my local machine. scraper:error Error: Protocol error (Target.setDiscoverTargets): Target closed. scraper:error at /home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/common/Connection.ts:102:57 scraper:error at new Promise (<anonymous>) scraper:error at Connection.send (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/common/Connection.ts:101:12) scraper:error at Function.create (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/common/Browser.ts:230:22) scraper:error at ChromeLauncher.launch (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/node/Launcher.ts:155:37) scraper:error at PuppeteerExtra.launch (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer-extra/src/index.ts:170:21) +0ms Error: Protocol error (Target.setDiscoverTargets): Target closed. at /home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/common/Connection.ts:102:57 at new Promise (<anonymous>) at Connection.send (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/common/Connection.ts:101:12) at Function.create (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/common/Browser.ts:230:22) at ChromeLauncher.launch (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer/src/node/Launcher.ts:155:37) at PuppeteerExtra.launch (/home/ec2-user/clanjobs-scripts/node_modules/puppeteer-extra/src/index.ts:170:21) scraper:error Error: Initialize timeout exceeded: 10000ms scraper:error at LinkedinScraper.<anonymous> (/home/ec2-user/clanjobs-scripts/node_modules/linkedin-jobs-scraper/build/scraper/LinkedinScraper.js:225:35) scraper:error at Generator.next (<anonymous>) scraper:error at fulfilled (/home/ec2-user/clanjobs-scripts/node_modules/linkedin-jobs-scraper/build/scraper/LinkedinScraper.js:5:58) scraper:error at runNextTicks (node:internal/process/task_queues:61:5) scraper:error at listOnTimeout (node:internal/timers:526:9) scraper:error at processTimers (node:internal/timers:500:7) +10s Setting chrome launch options { headless: true, args: [ '--enable-automation', '--start-maximized', '--window-size=1472,828', '--lang=en-GB', '--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage', '--disable-gpu', '--disable-accelerated-2d-canvas', '--disable-setuid-sandbox', '--disable-dev-shm-usage', "--proxy-server='direct://", '--proxy-bypass-list=', '--allow-running-insecure-content', '--disable-web-security', '--disable-client-side-phishing-detection', '--disable-notifications', '--mute-audio', '--start-maximized', '--window-size=1472,828', '--lang=en-GB', '--no-sandbox', '--disable-setuid-sandbox', '--disable-gpu', '--disable-dev-shm-usage', '--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage', "--proxy-server='direct://", '--proxy-bypass-list=', '--disable-accelerated-2d-canvas', '--disable-gpu', '--allow-running-insecure-content', '--disable-web-security', '--disable-client-side-phishing-detection', '--disable-notifications', '--mute-audio', '--enable-automation' ], defaultViewport: null, pipe: true, slowMo: 1000 }

spinlud commented 2 years ago

You can try to run your app in a docker container like: