issues
search
monperrus
/
crawler-user-agents
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:
MIT License
1.13k
stars
242
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
patterns json: add support for GroupMe bot
#359
bentsi
closed
1 month ago
2
Adding Monsidobot and removing whitespace
#358
theodorec2
closed
1 month ago
1
Add and use `requirements.txt`
#357
ericcornelissen
closed
2 months ago
1
Fix pattern of Nutch crawler
#356
haumacher
closed
2 months ago
1
Add ChatGPT-User - another bot from openai.com
#355
haumacher
closed
2 months ago
0
Add ClaudeBot
#354
haumacher
closed
2 months ago
0
golang: speedup using TRIE
#353
starius
opened
2 months ago
5
Bump tough-cookie and pacote
#352
dependabot[bot]
closed
2 months ago
0
Add npm dependencies with lockfile
#351
ericcornelissen
closed
2 months ago
4
Suggestion: test against false positives
#350
starius
closed
2 months ago
2
add date validation
#349
monperrus
closed
3 months ago
0
Add Golang package
#348
starius
closed
2 months ago
10
Use Node.js v20 in CI to have npm CLI version with support for --provenance
#347
ericcornelissen
closed
3 months ago
0
Update BuiltWith
#346
VoidMonk
closed
3 months ago
1
Add Audisto Crawler and PerplexityBot
#345
VoidMonk
closed
3 months ago
1
Add GeedoBot
#344
abushyk
closed
4 months ago
1
Add BuiltWith bot
#343
flopana
closed
4 months ago
1
port CI to GithubActions
#342
monperrus
closed
3 months ago
4
Add Expanse and InternetMeasurement crawler
#341
flopana
closed
4 months ago
1
Add PhxBot and ImagesiftBot
#340
dspinellis
closed
5 months ago
1
Add AI agents
#339
httpjamesm
opened
6 months ago
1
Escape dots in patterns
#338
giuscris
closed
2 months ago
5
Fix for SeznamBot detection
#337
synhershko
closed
7 months ago
1
Add DataForSeoBot
#336
abushyk
closed
8 months ago
1
added WordPress crawler
#335
petrokrupenia
closed
8 months ago
8
Add WordPress Bot/Crawler
#334
petrokrupenia
closed
6 months ago
2
Merge all yandex bots together
#333
fekir
closed
9 months ago
1
Add multiple bots found in the logs
#332
fekir
closed
9 months ago
7
Add Nicecrawler
#331
fekir
closed
9 months ago
1
Add Marginalia and YouBot
#330
dspinellis
closed
9 months ago
1
I noticed a recent bot which I believe is coming from Tencent but can't find documentation for - Ottabot
#329
daviddfm
opened
10 months ago
1
Add StractBot
#328
dspinellis
closed
10 months ago
1
Add patent crawler
#327
dspinellis
closed
10 months ago
0
Add WellKnownBot
#326
dspinellis
closed
10 months ago
1
add linespider
#325
monperrus
closed
10 months ago
0
miove CI to 3.11
#324
monperrus
closed
10 months ago
0
Add DataForSeoBot
#323
dspinellis
closed
10 months ago
0
Add current Turnitin instance
#322
dspinellis
closed
10 months ago
3
Add current AwarioBot instance
#321
dspinellis
closed
10 months ago
1
Add YandexRenderResourcesBot, LightspeedSystemsCrawler, ev-crawler, BitSightBot, woorankreview
#320
AlexanderChebanny
closed
10 months ago
1
Add GPTBot
#319
arch1v1st
closed
10 months ago
1
add google bots (Google-InspectionTool, Storebot-Google, GoogleOther)
#318
davgur
closed
11 months ago
1
Add Vk bot pattern (OdklBot)
#317
robinki
closed
1 year ago
1
Add Steam bot (SteamChatURLLookup)
#316
robinki
closed
1 year ago
1
fix: Fixed the path in main.php
#315
vanushwashere
closed
1 year ago
1
Fix an invalid pattern
#314
sidvishnoi
closed
1 year ago
1
Added composer.json and main.php
#313
vanushwashere
closed
1 year ago
4
Please also register this on packagist.org to use it via Composer
#312
vanushwashere
closed
1 year ago
5
Add Google-Read-Aloud
#311
ictran
closed
1 year ago
1
Add linespider
#310
Almenon
closed
10 months ago
3
Next