alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
24 stars 11 forks source link

Handle WebMeUp bots (BLEXBot) #158

Closed gabrieljablonski closed 2 months ago

gabrieljablonski commented 2 months ago

BLEXBot is a crawler that is responsible for a good chunk of bot requests on a website I manage (>200k requests over multiple IPs), so handling it with legitbot seems like a good idea.

The bot specs are available here: http://webmeup-crawler.com/

Let me know if any changes are needed.

gabrieljablonski commented 2 months ago

Btw, sorry for spamming PRs.

I'm going over some IP addresses that are littering user tracking in my project, so I'm submitting for each of the top crawler bots that aren't already covered by legitbot.

This should be the last one for now. Thanks!

alaz commented 2 months ago

Btw, sorry for spamming PRs.

No, no, your PRs are perfect. I do prefer to have separate commits for bots.

codecov-commenter commented 2 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.77%. Comparing base (13d53f6) to head (e5ad758). Report is 1 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #158 +/- ## ========================================== + Coverage 98.75% 98.77% +0.02% ========================================== Files 25 26 +1 Lines 240 245 +5 ========================================== + Hits 237 242 +5 Misses 3 3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

alaz commented 2 months ago

This should be the last one for now

I am a bit tied up at the moment, but will make a release this weekend latest.

gabrieljablonski commented 2 months ago

I am a bit tied up at the moment, but will make a release this weekend latest.

No worries. Thanks for taking a look so quickly!