search

monperrus / crawler-user-agents

Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:
MIT License
1.17k stars 250 forks source link

Add more users agents of bots found in the wild #264

Closed komapa closed 4 years ago

komapa commented 4 years ago

These are real-world bots that we see daily and we have added manually to account for the fact that they are not present in this repo. I hope this is helpful for others as well.

monperrus commented 4 years ago

Thanks for your contribution. There are problems, see CI logs.

komapa commented 4 years ago

Should be ready for review again. Thanks!

komapa commented 4 years ago

Bump!

monperrus commented 4 years ago

Thanks a lot!