The same bot is listed multiple times. - Githubissues

fabiomb / is_bot

Simple PHP function to detect search engines bots

GNU General Public License v2.0

62 stars 10 forks source link

The same bot is listed multiple times. #12

Closed webgoto closed 1 year ago

webgoto commented 1 year ago

Thanks for the great list. It looks like the same bot is listed multiple times.

name	times
MJ12bot	3
Barkrowler	3
Baiduspider	2
R6_FeedFetcher	2
Sogou web spider	2
AhrefsBot	2
Exabot	2
Ezooms	2
SearchmetricsBot	2
magpie-crawler	2
GrapeshotCrawler	2
Spinn3r	2
linkdexbot	2
AdlesseBot	2
MegaIndex.ru	2
ltx71	2
CCBot	2
Mediatoolkitbot	2
SemrushBot	2
DomainCrawler	2
archive.org_bot	2
semantic-visions.com	2
FemtosearchBot	2
Cliqzbot	2
SEOkicks	2
ZoominfoBot	2
Linguee Bot	2
Mail.RU_Bot	2
Nutch	2
scalaj-http	2
serpstatbot	2
adscanner	2
BacklinkCrawler	2
netEstate NE Crawler	2
DomainStatsBot	2
Scrapy	2
Blackboard	2
Domains Project	2
SerendeputyBot	2
Cocolyzebot	2
LinkpadBot	2
CensysInspect	2
MauiBot	2
CriteoBot	2
heritrix	2
Buck	2

fabiomb commented 1 year ago

thanks!

fabiomb commented 1 year ago

fixed duplicates