search

sifatul / codies

An open source project to track profiles of developers and show their info in a single page so that recruiters/peer developers do not have to visit many different platform
https://find-profile.vercel.app/
MIT License
15 stars 14 forks source link

find public/open source api for linkedin #3

Open sifatul opened 2 years ago

Zishan3165 commented 2 years ago

Currently LinkedIn does not provide their data publicly since 2015 (People Search API). The only way to get official access to their search APIs is apply for developer partner from which we can get the respective auth tokens etc to fetch their data. However, I'm not sure if this is safe, since everyone would use those credentials to make searches. We might have to look into open source scrappers for LinkedIn. @Sifatul

sifatul commented 2 years ago

Currently LinkedIn does not provide their data publicly since 2015 (People Search API). The only way to get official access to their search APIs is apply for developer partner from which we can get the respective auth tokens etc to fetch their data. However, I'm not sure if this is safe, since everyone would use those credentials to make searches. We might have to look into open source scrappers for LinkedIn. @Sifatul

@Zishan3165 how about creating a feature where we show LinkedIn data from our database, but if data is not in the database then users can log in with their LinkedIn credentials to view data from LinkedIn through our system.

In that case, it's going to be safe and with the user's consent we can do the following:

  1. like true caller , we can store their own information and searched information in our DB
  2. if any other user searches a similar profile, we can show from our database

Possible ways of scrapping:

  1. https://github.com/joeyism/linkedin_scraper
  2. https://github.com/linkedtales/scrapedin
nsourov commented 2 years ago

@Sifatul I have a repository for LinkedIn profile scraper https://github.com/nsourov/linkedin-profile-scraper. I am not sure this is still working, but the logic I implemented is that instead of using a direct password, the user can send cookies. This way the scraper gets data faster because It doesn't need to login.

In our case, we can use a dummy account cookie so that users don't need to pass their username and password as It's very confidential data.

sifatul commented 2 years ago

@nsourov this won't be a viable option. As many users will use our platform that dummy account will get blocked within a week. Instead of being a feature it might become a liability for us :D So, need to find an alternative