search

varnishcache / varnish-devicedetect

VCL based device detection for Varnish Cache.
Other
300 stars 88 forks source link

Need to differentiate between mobile bot and desktop bot #18

Closed dashfwd closed 10 years ago

dashfwd commented 10 years ago

Right now all bots are grouped together into a single X-UA-Device of 'bot'. But Google, for example, has bots that are specifically used for crawling mobile sites. See:

https://developers.google.com/webmasters/smartphone-sites/googlebot-mobile http://googlewebmastercentral.blogspot.com/2014/01/a-new-googlebot-user-agent-for-crawling.html

I'm not sure of whether there are other "mobile bot" user agents, but it might make sense to have a 'mobile-bot' X-UA-Device.

lkarsten commented 10 years ago

Hi. This sounds reasonable, and should be a simple change. Submit a pull request and I'll have a look at it.

pippoppo commented 10 years ago

Hi, this morning I made a very quick and very dirty patch to address this issue, like this

            if (req.http.User-Agent ~ "(?i)(ads|google|bing|msn|yandex|baidu|ro|career|)bot" ||
                req.http.User-Agent ~ "(?i)(baidu|jike|symantec)spider" ||
                req.http.User-Agent ~ "(?i)scanner" ||
                req.http.User-Agent ~ "(?i)(web)crawler") {
                    if (req.http.User-Agent ~ "iPhone; CPU iPhone") {
                            set req.http.X-UA-Device = "mobile-bot";
                            } else {
                            set req.http.X-UA-Device = "bot";
                            }
                    }

If you're interested I can arrange a pull request for you and the OP

lkarsten commented 9 years ago

For future reference, this was added in e59973798613a5c7f5472be1c9788dd5d01cc0c1.