search

propublica / upton

A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)
MIT License
1.62k stars 113 forks source link

Warn users of slug collisions #27

Open jeremybmerrill opened 10 years ago

jeremybmerrill commented 10 years ago

Keep an in-memory Hash of slugs to URLs. If one is found, warn the user.

This will be imperfect (if the colliding URLs are stashed at separate runs of Upton), but that's okay.