alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
24 stars 11 forks source link

Handle DataForSEO bots #157

Closed gabrieljablonski closed 2 months ago

gabrieljablonski commented 2 months ago

DataForSEO is a crawler that is responsible for >50% of bot requests on a website I manage (>1.3M requests from a single IP address in the past few months), so handling it with legitbot seems like a good idea.

The bot specs are available here: https://dataforseo.com/dataforseo-bot

Let me know if any changes are needed.

codecov-commenter commented 2 months ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.75%. Comparing base (b4d6dce) to head (24ec3c3). Report is 1 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #157 +/- ## ========================================== + Coverage 98.72% 98.75% +0.02% ========================================== Files 24 25 +1 Lines 235 240 +5 ========================================== + Hits 232 237 +5 Misses 3 3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.