We already visited some websites. The visited URLs are all stored at a tiny sqlite3 db. We should build a revisit policy to know if there is any update on the data, like:
save the last visited at the sqlite3 db
list URLs which is not parsed or last visited time is longer than 3 month
We already visited some websites. The visited URLs are all stored at a tiny sqlite3 db. We should build a
revisit policyto know if there is any update on the data, like: