Strange user agent ? Serf/1.1.0 mod_pagespeed/1.9.32.3-4448

[vinvin27]

Hello,

I got lot of access from this user agent Serf/1.1.0 mod_pagespeed/1.9.32.3-4448 and the IP is my server's IP..

My following pagespeed config :

pagespeed on;
        pagespeed FileCachePath /var/ngx_pagespeed_cache;
        # disable CoreFilters
        pagespeed RewriteLevel PassThrough;

        # enable collapse whitespace filter
        pagespeed EnableFilters collapse_whitespace;

        # enable JavaScript library offload
        pagespeed EnableFilters canonicalize_javascript_libraries;

        # combine multiple CSS files into one
        pagespeed EnableFilters combine_css;

        # combine multiple JavaScript files into one
        pagespeed EnableFilters combine_javascript;

        # remove tags with default attributes
        pagespeed EnableFilters elide_attributes;

        # improve resource cacheability
        pagespeed EnableFilters extend_cache;

        # flatten CSS files by replacing @import with the imported file
        pagespeed EnableFilters flatten_css_imports;
        pagespeed CssFlattenMaxBytes 5120;

        # defer the loading of images which are not visible to the client
        pagespeed EnableFilters lazyload_images;

        # enable JavaScript minification
        pagespeed EnableFilters rewrite_javascript;

        # enable image optimization
        pagespeed EnableFilters rewrite_images;

        # pre-solve DNS lookup
        pagespeed EnableFilters insert_dns_prefetch;

        # rewrite CSS to load page-rendering CSS rules first.
        pagespeed EnableFilters prioritize_critical_css;

Bad cache ? Or something else ?

Thanks :)

[oschaaf]

@vinvin27 The user-agent you mention is used by ngx_pagespeed for fetching resources internally, and is expected. You could consider LoadFromFile [1] as an alternate means for loading resources. Normally there would be a short period with lots of these requests as pagespeed discovers the website, which will settle down. If you see large amounts of these requests continuously you might have to:

• increase the expiry of the original css/js/images (or use LoadFromFile)
• increase the cache size. It could be that ngx_pagespeed's assigned cache size is not large enough. You can check cache hit ratios and resource fetch counts via the admin pages [2].

[1] https://developers.google.com/speed/pagespeed/module/domains#ModPagespeedLoadFromFile [2] https://developers.google.com/speed/pagespeed/module/admin

[MiguelQueiroz]

yes but for example in my case it is trying to fetch wrong locations.. for example i have tons of this:

10.0.0.X - - [26/Sep/2016:02:41:20 +0000] "GET /logo_sticky.png HTTP/1.1" 404 577 "http://beautivmazure.cloudapp.net/" "Serf/1.1.0 mod_pagespeed/1.9.32.14-0" *0* they are error 404 ! and plus /loco_sticky.png does not exists on my domain, because it is a secundary cloud CDN images provider.. And that domain beautivmazure.cloudapp.net particularly should not be used, because it uses our generic template which lacks translations, etc, and many other things, that wont connect correctly, including link placeholders, thats why the error 404 ... How can i stop this all together, or exclude this domain ( the own domain)

[oschaaf]

@MiguelQueiroz It is possible to restrict optimization, so for example you could add the following to your configuration to exclude beautivmazure.cloudapp.net:

pagespeed Disallow "http://beautivmazure.cloudapp.net/*";

See https://developers.google.com/speed/pagespeed/module/restricting_urls Does that help?

[MiguelQueiroz]

yes thanks. that helps.