Closed alanorth closed 4 years ago
This adds patterns to match the following robot user agents:
Relevant URLs for each user agent included in the patterns file.
Note: I had added a few of these in #34 but only a few were merged. I will re-submit them again here.
This adds patterns to match the following robot user agents:
Relevant URLs for each user agent included in the patterns file.
Note: I had added a few of these in #34 but only a few were merged. I will re-submit them again here.