gt-big-data / QDoc

Quick & Dirty Operating Crawler
4 stars 1 forks source link

Don't overwrite feed URLs. #33

Closed supersam654 closed 8 years ago

supersam654 commented 8 years ago

We currently overwrite URLs for articles and feeds. Turns out that we shouldn't try doing this with feeds.

After crawling, we basically do something like: self.feed = request.url which we actually shouldn't do.