search

nirbarazida / Data-mining-project

ITC - Data mining project
0 stars 1 forks source link

last user in the website #7

Closed InbarShirizly closed 4 years ago

InbarShirizly commented 4 years ago

add a feature that solves the problem when trying to scrap over the last user in the website:

  1. add value in the website table: total_users
  2. check and stop scrapping for the specific website if exceeds this number
InbarShirizly commented 4 years ago

This feature is not important because the program runs from the first users and not designed to scrap more the 100000 users (this is the bare minimum of users in each of the big websites)