search

commoncrawl / news-crawl

News crawling with StormCrawler - stores content as WARC
Apache License 2.0
316 stars 34 forks source link

Route tuples to the status updater bolt based on URLs #65

Closed jnioche closed 8 months ago

jnioche commented 8 months ago

See #63

When using more than one instance of the status updater bolt, the tuples need to be routed to the instance based on the URL so that the caching can be leveraged.