crawler for researchgate page

conferency / find-my-reviewers

Find-my-reviewers matches scholars and paper together with topic extraction (LDA).

http://FindMyReviewers.com

MIT License

12 stars 9 forks source link

Open harrywang opened 7 years ago

harrywang commented 7 years ago

we need to create a crawler to get all information if a user provides his/her researchgate.

I think for this type of crawling (a few pages only) - beautiful soup might be better than scrapy

check the following sample pages on what information could be there and should be collected:

harrywang commented 7 years ago

keep it simple for now - just information from the profile page - one page only for the first version.