Closed summercms closed 3 years ago
See github issue: #540
This repo is missing one of the regex - there are two bots (not one).
CCBot/2.0 (http://commoncrawl.org/faq/)
CCResearchBot/1.0 commoncrawl.org/research//Nutch-1.7-SNAPSHOT
Link: http://commoncrawl.org/
Coverage increased (+0.02%) to 99.97% when pulling ae39b1d145e423d28e9df9a0b92b886634906e0d on ayumi-cloud:CommonCrawl into 880b9fa797401d14b28956442944c3daa70240ff on WhichBrowser:master.
Fixed in bad3d4004e308ac5fd941a228f57abb507190ac9
See github issue: #540
This repo is missing one of the regex - there are two bots (not one).
Common Crawl Bot
Common Crawl Research Bot
Link: http://commoncrawl.org/