Closed emanruse closed 6 years ago
Thanks @emanruse do you have a better regex for this? I do not use Apache anymore myself but still support it and would like to keep this running for others.
RewriteCond %{HTTP_USER_AGENT} ^(.*)(\bDotbot\b).* [NC,OR]
Perhaps even this would work (as the leading and trailing .*
really only burden the regex):
RewriteCond %{HTTP_USER_AGENT} \bDotbot\b [NC,OR]
Sorry for the delayed response on this been so busy with other projects. Will test this out today and tag you so you can test changes.
@mitchellkrogza I can confirm from testing that @emanruse \b<STRING>\b
regex works
Thanks @davcpas1234 I will push out a modified a fix now now so you and @emanruse can test.
@davcpas1234 @emanruse please can you test latest updates
Also
should we not improve the regex for %{HTTP_REFERER} too or is that working ok?
ie.
RewriteCond %{HTTP_REFERER} ^http(s)?://(www.)?.*prizestohandle\.club.*$ [NC,OR]
to
RewriteCond %{HTTP_REFERER} \bprizestohandle\.club\b [NC,OR]
@mitchellkrogza - Happy to confirm solution from a bot perspective:
In regards to %{HTTP_REFERER}
suggestion, the current format works well from my perspective as it eliminates sub-domains and sub-directories.
Thanks so much David that's great we will stick with the current format. Appreciate you testing and helping to improve things for the project.
This:
will not match:
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider"
This:
will not match: