search

linkedtales / scrapedin-linkedin-crawler

Crawler for LinkedIn full profiles 2019
Apache License 2.0
215 stars 71 forks source link

linkedin account was banned after I use this scrapper #17

Closed tszjqgs closed 5 years ago

tszjqgs commented 5 years ago

after I got several result , the account was banned from linkedin , so how to avoid this happen if I register another account ?

tszjqgs commented 5 years ago

and I referenced your reply on scrapedin , there was an waitingtime parameter for that . but for scraping multiple profiles , I use this scrapedin-linkedin-crawler , I did not find the same parameter in the src js files yet . could someone give some hint to deroute this issue? thanks

tszjqgs commented 5 years ago

I just saw the profile wait time parameter in conf.json file , I will try enlarge that one

leonardiwagner commented 5 years ago

Hi @tszjqgs

after I got several result , the account was banned from linkedin , so how to avoid this happen if I register another account ?

Honestly, I don't know how, because it's something on LinkedIn that detects the account usage and decides to block, and I'm unaware of that logic.

I suppose that's something like watching hundreds of profiles, or due to very fast clicking, so you could try:

Please try these options and give us some feedback.

tszjqgs commented 5 years ago

thanks , enlarge that parameter helped

tszjqgs commented 5 years ago

thanks, i submit the id and see what can be done after that ,still waiting response from linkedin

发件人：Wagner Leonardi notifications@github.com 发送时间：2019年10月17日(星期四) 22:50 收件人：linkedtales/scrapedin-linkedin-crawler scrapedin-linkedin-crawler@noreply.github.com 抄　送：tszjqgs jason4ever8@aliyun.com; Mention mention@noreply.github.com 主　题：Re: [linkedtales/scrapedin-linkedin-crawler] linkedin account was banned after I use this scrapper (#17)

Hi @tszjqgs after I got several result , the account was banned from linkedin , so how to avoid this happen if I register another account ? Honestly, I don't know how, because it's something on LinkedIn that detects the account usage and decides to block, and I'm unaware of that logic. I suppose that's something like watching hundreds of profiles, or due to very fast clicking, so you could try: increase wait time seconds, to decrease the number of viewed profiles per seconds remove that line on /node_modules/scrapedin/profile.js so there will be no clicking on the page (however you'll collect less information) you can also remove that line as well that scrolls on the page, maybe they're listening to the scroll event as well I'm also not sure but logging via cookies maybe helps too. Maybe requiring them to unblock your account (proving with an Id or something they request) you could have an "unlimited" account, I never did that before, but you can also try. Please try these options and give us some feedback. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

tszjqgs commented 5 years ago

and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping)

leonardiwagner commented 5 years ago

I updated now the project to you place your cookies file in config.json see changes here: and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping)

you can either git pull or apply those changes manually in your code

tszjqgs commented 5 years ago

thanks, I redid the git clone over again and will try it out

发件人：Wagner Leonardi notifications@github.com 发送时间：2019年10月18日(星期五) 22:24 收件人：linkedtales/scrapedin-linkedin-crawler scrapedin-linkedin-crawler@noreply.github.com 抄　送：tszjqgs jason4ever8@aliyun.com; Mention mention@noreply.github.com 主　题：Re: [linkedtales/scrapedin-linkedin-crawler] linkedin account was banned after I use this scrapper (#17)

I updated now the project to you place your cookies file in config.json see changes here: and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping) you can either git pull or apply those changes manually in your code — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

tszjqgs commented 5 years ago

hi wangner , I installed the Editthiscookie extension in browser , and loggedin the linkedin , than I click the cookie button , click the export , and directly copy the content into the config.json at the cookie area , but the npm start can not start welll, could you please provide a user guide about the coockie useage ? thanks

发件人：Wagner Leonardi notifications@github.com 发送时间：2019年10月18日(星期五) 22:24 收件人：linkedtales/scrapedin-linkedin-crawler scrapedin-linkedin-crawler@noreply.github.com 抄　送：tszjqgs jason4ever8@aliyun.com; Mention mention@noreply.github.com 主　题：Re: [linkedtales/scrapedin-linkedin-crawler] linkedin account was banned after I use this scrapper (#17)

I updated now the project to you place your cookies file in config.json see changes here: and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping) you can either git pull or apply those changes manually in your code — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

leonardiwagner commented 5 years ago

Hi @tszjqgs I'm closing this issue because there's a lot of questions on the same issue and I'm getting lost trying to answer all of them.

Please open another issue if you still have some problem, thank you.