Use upsert for MongoDB - Githubissues

BIDS-projects / scraper

Collects data from websites of data science institutions

2 stars 0 forks source link

Use upsert for MongoDB #27

Closed don-han closed 8 years ago

don-han commented 8 years ago

This prevents

re-scraping HTML from already visited urls, reducing # of I/Os
inserting duplicates