search

headrun / SWIFT

2 stars 0 forks source link

Crawler to fetch Job Details #106

Open SreenivasDega opened 4 years ago

SreenivasDega commented 4 years ago

To Write a crawler which helps in getting the data from google for the below openings.

PHP developer Jave Developer

Data points to be captured: Data points required Company Name Functional Area Industry Education Experience Required Skills required Location When posted (TIme stamp) Job description

jaffrinkirthiga96 commented 4 years ago

Have crawled the Job Postings for java and PHP developers and shared the data in csv and json files.

jaffrinkirthiga96 commented 4 years ago

Have analysed naukri.com and got the API to get the job posts details. API: headers = { 'authority': 'www.naukri.com', 'x-newrelic-id': 'VgUHWVFXABACXVdaBAgHUF0=', 'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36', 'systemid': '109', 'content-type': 'application/json', 'accept': 'application/json', 'clientid': 'd3skt0p', 'cache-control': 'no-cache', 'appid': '109', 'sec-fetch-site': 'same-origin', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', 'referer': 'https://www.naukri.com/php-developer-jobs-in-banglore-2?k=php%20developer&l=banglore&cityType=25.17.74', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8' } params = ( ('noOfResults', '20'), ('urlType', 'search_by_key_loc'), ('searchType', 'adv'), ('keyword', 'php developer'), ('location', 'banglore'), ('pageNo', '2'), ('sort', 'r'), ('cityType', ['25.17.74', '25.17.74']), ('k', 'php%20developer'), ('l', 'banglore'), ('seoKey', 'php-developer-jobs-in-banglore-2'), ('src', 'jobsearchDesk'), ('latLong', '') ) response = requests.get('https://www.naukri.com/jobapi/v3/search', headers=headers, params=params)