search

tomquirk / linkedin-api

👨‍💼 LinkedIn API for Python
https://pypi.org/project/linkedin-api
MIT License
1.86k stars 416 forks source link

Banned accounts fast #127

Closed mbryde closed 5 months ago

mbryde commented 3 years ago

Hello,

I have tried this scraper for a short period of time, and every day even after just looking at a maximum of two people using a single account it get's "banned" / login restricted. So I need to supply a valid ID to unlock the account again. What are you guys experience with getting accounts banned?

Sinu5oid commented 3 years ago

Hi. Probably you are getting so-called ChallengeException (which is 401 with payload CSRF check failed) Check this tool out: https://github.com/everping/Linkedin-Authentication-Challenge

mbryde commented 3 years ago

@Sinu5oid It's not because of any challenges. I have manually created the account in a browser and know it's working. I get no CSRF fails. But my accounts get banned after around 24 hours. Even tho I might just scrape a single persons info.

sliao7 commented 3 years ago

I have the same issue. My account was banned immediately. Not sure if it is because they detect my ip address and banned any account from that ip address.

diegofcoelho commented 3 years ago

I prefer the turtle path: I set scheduled tasks with randomly spaced runtimes and have been running it for weeks now.. :)

sliao7 commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

diegofcoelho commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

sliao7 commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

diegofcoelho commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

Much longer! Perhaps 1-2 profile scrapes per minute? Think how long you would take to get that info manually and work around that

sliao7 commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

Much longer! Perhaps 1-2 profile scrapes per minute? Think how long you would take to get that info manually and work around that

I see. Thank you so much for your reply! Happy new year!

sliao7 commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

Much longer! Perhaps 1-2 profile scrapes per minute? Think how long you would take to get that info manually and work around that

I see. Thank you so much for your reply! Happy new year!

One more thing. Is there an approximate daily limit for the number of scrapes? Or approximately how many profiles can I get per day?

diegofcoelho commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

Much longer! Perhaps 1-2 profile scrapes per minute? Think how long you would take to get that info manually and work around that

I see. Thank you so much for your reply! Happy new year!

One more thing. Is there an approximate daily limit for the number of scrapes? Or approximately how many profiles can I get per day?

I never heard of a set number as a daily limit for any of the sections @sliao7! And Happy New Year too!!

sliao7 commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

Much longer! Perhaps 1-2 profile scrapes per minute? Think how long you would take to get that info manually and work around that

I see. Thank you so much for your reply! Happy new year!

One more thing. Is there an approximate daily limit for the number of scrapes? Or approximately how many profiles can I get per day?

I never heard of a set number as a daily limit for any of the sections @sliao7! And Happy New Year too!!

Hey, @diegofcoelho , are you using your personal Linkedin account to scrape? When I use a dummy account without any connection, it gets banned quickly and Linkedin requires me to upload my passport to verify my identity.

diegofcoelho commented 3 years ago

I prefer the turtle path: I set scheduled paths with randomly spaced runtimes and have been running it for weeks now.. :)

Wow! Did you use rotated proxies and clear the cookies?

Not really, but mostly because I am not using many accounts. You can pickle and store the session to avoid logging in too often. I feel like rotating cookies may also trigger LinkedIn that someone is trying to log to the account (like hacking).

Got it! Thank you so much! I tried to set around 1min random sleeping times and was blocked after 50 scrapes. Are you using some longer sleeping times?

Much longer! Perhaps 1-2 profile scrapes per minute? Think how long you would take to get that info manually and work around that

I see. Thank you so much for your reply! Happy new year!

One more thing. Is there an approximate daily limit for the number of scrapes? Or approximately how many profiles can I get per day?

I never heard of a set number as a daily limit for any of the sections @sliao7! And Happy New Year too!!

Hey, @diegofcoelho , are you using your personal Linkedin account to scrape? When I use a dummy account without any connection, it gets banned quickly and Linkedin requires me to upload my passport to verify my identity.

Yes, mine and from the part interested in the data.

lavarthan commented 3 years ago

I'm also having the same problem my account is getting banned so quickly (~50 profile scrapes ). I once scraped around 3000 at a time without getting blocked with a new account. But now it seems impossible to pass the 50 profile. are they tracking IP addresses and blocking? I also tried to google colab but logging in with colab is impossible (since google colab is having some set range of IP address) if anybody finds the solution, please post