Black listing in DBW batch-gen

scrapinghub / frontera

A scalable frontier for web crawlers

BSD 3-Clause "New" or "Revised" License

1.29k stars 216 forks source link

Open sibiryakov opened 6 years ago

sibiryakov commented 6 years ago

Please remove reference to bigbot_common

Prometheus3375 commented 4 years ago

I thinks this PR can be closed, if Frontera should not use bigbot_common.