linkedtales / scrapedin

LinkedIn Scraper (currently working 2020)
Apache License 2.0
588 stars 171 forks source link

manual login is working fine without verification unable to scrape #144

Closed pwnmeow closed 1 year ago

pwnmeow commented 3 years ago

manual login is working fine without verification

(node:10922) UnhandledPromiseRejectionWarning: Error: linkedin: manual check was required, verify if your login is properly working manually or report this issue: scrapedin 1.0.21 https://github.com/linkedtales/scrapedin/issues at /root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/scrapedin/src/login.js:63:31 at processTicksAndRejections (internal/process/task_queues.js:97:5) at async module.exports (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/scrapedin/src/scrapedin.js:29:7) (node:10922) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag--unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2) (node:10922) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. (node:10922) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.evaluate): Target closed. at /root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/Connection.js:183:56 at new Promise (<anonymous>) at CDPSession.send (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/Connection.js:182:12) at ExecutionContext.evaluateHandle (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/ExecutionContext.js:72:75) at ExecutionContext.<anonymous> (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/helper.js:109:23) at /root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/DOMWorld.js:125:38 at processTicksAndRejections (internal/process/task_queues.js:97:5) -- ASYNC -- at Frame.<anonymous> (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/helper.js:108:27) at Page.$ (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/Page.js:300:29) at Page.<anonymous> (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/puppeteer/lib/helper.js:109:23) at /root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/scrapedin/src/login.js:61:16 at processTicksAndRejections (internal/process/task_queues.js:97:5) at async module.exports (/root/Desktop/HelperScriptsHackingArchivesOfIndia/linkedinscraped/node_modules/scrapedin/src/scrapedin.js:29:7) (node:10922) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag--unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)

pwnmeow commented 3 years ago

my script was working fine until 2 minutes ago its not a connection issue I also tried changing the account but no luck

leonardiwagner commented 3 years ago

@sheerazali19 as it is stated Error: linkedin: manual check was required , see how to pass that manual check here and using cookies to login usually by pass that manual check.

victhor9696 commented 3 years ago

I'm having the same problem, but when I try to perform the scrap via cookie with isHeadless: false, return this error, did anyone manage to help me?

it's like something is missing

(node:30902) UnhandledPromiseRejectionWarning: Error: Failed to launch chrome!

(chrome:30913): Gtk-WARNING **: 22:58:24.646: cannot open display:

TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

at onClose (/scrapedin/node_modules/puppeteer/lib/Launcher.js:342:14)
at ChildProcess.helper.addEventListener (scrapedin/node_modules/scrapedin/node_modules/puppeteer/lib/Launcher.js:332:60)
at ChildProcess.emit (events.js:203:15)
at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)

(node:30902) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

pwnmeow commented 3 years ago

Well i used my script and logged in to 3 different accounts when one showed captcha on a loop that solved it for me

Bilal815 commented 1 year ago

I recently tried Crawlbase to scrape LinkedIn and it worked like a charm. I have gone through their docs, it says that you need JS enabled to scrape/crawl dynamic sites. This is where it gets interesting https://crawlbase.com/docs/crawling-api/scrapers/#linkedin. Just make sure to signup.