search

headrun / SWIFT

2 stars 0 forks source link

Yoket.in Crawler #86

Closed SreenivasDega closed 4 years ago

SreenivasDega commented 4 years ago

To Crawl the candidate profiles and share it to the customer

Preethi266 commented 4 years ago

Previously we are signin with the google so while singnin it is showing like please signin with google.So Yesterday i have created another login account for this.

Preethi266 commented 4 years ago

Yesterday i have taken the available data in the site. By doing more requests iam getting the 429 response .So today i will try with the rotating proxies for the requests. Screenshot from 2020-06-10 16-54-32

Preethi266 commented 4 years ago

Yesterday i have rotated the proxies for the spider.I need to keep more proxies for this site because they identifying the proxies also for multiple requests.And i have shared the sample data.

Preethi266 commented 4 years ago

Tried with the login details i am getting the experience and tofel score and populated the data.I have taken the sample data.

malla794 commented 4 years ago

@vinuthnaPathri / @sunil-headrun,

We have the completed runs for the yocket website crawling and here I'm sharing the master data. Please check once.

https://drive.google.com/file/d/15bVY8_uiMLIc9k-iFf2czx_YSZyh5LEA/view?usp=sharing

vinuthnaPathri commented 4 years ago

@malla794 Sunil asked this to be validated by Harsha . Please get this done and confirm any gaps are to be fixed.