linkedtales / scrapedin

LinkedIn Scraper (currently working 2020)
Apache License 2.0
597 stars 174 forks source link

(node:2779) UnhandledPromiseRejectionWarning: Error: linkedin: manual check was required, #124

Closed EmanFateen closed 4 years ago

EmanFateen commented 4 years ago

Hello, I'm working on a project which uses scraped to make crawling on LinkedIn accounts, It's working fine on my localhost on my local machine, But after uploaded the project it to a dedicated server, it started to throughout this error on the consol

Browser started scrapedin: 2020-07-07T10:20:56.346Z info: [scrapedin.js] initializing scrapedin: 2020-07-07T10:20:56.458Z info: [scrapedin.js] email and password was provided, we're going to login... scrapedin: 2020-07-07T10:20:56.815Z info: [login.js] logging at: https://www.linkedin.com/login scrapedin: 2020-07-07T10:21:12.069Z warn: [login.js] successful login element was not found scrapedin: 2020-07-07T10:21:12.074Z warn: [login.js] manual check was required (node:2779) UnhandledPromiseRejectionWarning: Error: linkedin: manual check was required, verify if your login is properly working manually or report this issue: scrapedin 1.0.18 https://github.com/linkedtales/scrapedin/issues at /home/dummy/test.ai/searcher/node_modules/scrapedin/src/login.js:63:31 at processTicksAndRejections (internal/process/task_queues.js:97:5) at async module.exports (/home/dummy/test.ai/searcher/node_modules/scrapedin/src/scrapedin.js:21:7) at async Function.init (/home/dummy/test.ai/searcher/scraper.js:11:28) (node:2779) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2) (node:2779) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. (node:2779) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.evaluate): Target closed. at /home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/Connection.js:183:56 at new Promise () at CDPSession.send (/home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/Connection.js:182:12) at ExecutionContext.evaluateHandle (/home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/ExecutionContext.js:72:75) at ExecutionContext. (/home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/helper.js:109:23) at /home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/DOMWorld.js:125:38 at processTicksAndRejections (internal/process/task_queues.js:97:5) -- ASYNC -- at Frame. (/home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/helper.js:108:27) at Page.$ (/home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/Page.js:300:29) at Page. (/home/dummy/test.ai/searcher/node_modules/scrapedin/node_modules/puppeteer/lib/helper.js:109:23) at /home/dummy/test.ai/searcher/node_modules/scrapedin/src/login.js:61:16 at processTicksAndRejections (internal/process/task_queues.js:97:5) at async module.exports (/home/dummy/test.ai/searcher/node_modules/scrapedin/src/scrapedin.js:21:7) at async Function.init (/home/dummy/test.ai/searcher/scraper.js:11:28) (node:2779) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)

It couldn't crewel anymore! I'm using a dummy linked account which I just created 3 days ago and these are my settings

searchingData: {
        email: 'dummy@domian.com',
        password: 'secret#',
        hasToLog: true,
        isHeadless: true,
        puppeteerArgs: {
            args: [
                '--no-sandbox'
            ]
        },
        interval: 13000,
    }

const profileScraper = await scrapedin(searchingData)

I working very fine on the localhost. but doesn't work on the liver server.

Is there any change I should make in the LinkedIn account to allow the scrapedin use this account ??

BTW It sends a code to my email but I don't know where I should enter this code.

Thank you.

gautierdag commented 4 years ago

Since you are running it on a server (headless mode), you should probably use cookies. You will need to refresh (regenerate) the cookies on your local machine if you are blocked however.

EmanFateen commented 4 years ago

@gautierdag Thank you It worked now (Y)