search

4pr0n / gonewilder

GNU General Public License v2.0
32 stars 11 forks source link

Backfill user #5

Open 4pr0n opened 10 years ago

4pr0n commented 10 years ago

Say we add support for a new domain/host.

We'll want to rescrape everyone to get the missed content.

No way to do that if using 'last post id' when scraping users.

jadedgnome commented 10 years ago

scrape the history.log files 'domain not supported' on the next scraping iteration for all users and subsequent new users. choice is up to you, if you want to look through history.log files for users marked deleted.