search

apache / incubator-pagespeed-mod

Apache module for rewriting web pages to reduce latency and bandwidth.
http://modpagespeed.com
Apache License 2.0
697 stars 158 forks source link

Don't reinject ?ModPagespeed=noscript when the origin URL has ?modpagespeed=noscript #1024

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
According to the user, this results in googlebot sending requests like this:

http://www.tophebergeur.com/tutoriels/cloud/infrastructure_cloud/?modpagespeed=n
oscript&pagespeed=noscript&pagespeed=noscript&pagespeed=noscript&pagespeed=noscr
ipt&pagespeed=noscript&pagespeed=noscript...

Original message:

Hello, 
I was analyzing the server log and I noticed that googlebot is crawling such 
urls  
/tutoriels/joomla/2/?modpagespeed=noscript&pagespeed=noscript&pagespeed=noscript
&pagespeed=noscript&pagespeed=noscript&pagespeed=noscript&pagespeed=noscript ; 
http://www.tophebergeur.com/tutoriels/cloud/infrastructure_cloud/?modpagespeed=n
oscript&pagespeed=noscript&pagespeed=noscript&pagespeed=noscript&pagespeed=noscr
ipt&pagespeed=noscript&pagespeed=noscript and so on.... And whee I do such a 
search site:www.tophebergeur.com  I see that Google is indexing such urls which 
can cause duplicate content and hurt website,s SEO. 
Can you please tell me how to fix this issue 

thanks a lot

Original issue reported on code.google.com by jmara...@google.com on 23 Nov 2014 at 7:57

eldk commented 8 years ago

Hello,

I've read this : https://groups.google.com/forum/#!searchin/mod-pagespeed-discuss/Googlebot/mod-pagespeed-discuss/vflZs5FEDdc/fgy6EOBMOPYJ

And have the same case, Googlebot is exploring URLS with more and more &pagespeed=noscript .

This is due to the use of Defer Javascript.

Will look to see what is the origin of those urls.

Greatings,

Eric

PS : I use canonical urls

X-Mod-Pagespeed:1.9.32.4-7251

Version: 13: on

Filters cw Collapse Whitespace jc Combine Javascript gp Convert Gif to Png jp Convert Jpeg to Progressive jw Convert Jpeg To Webp pj Convert Png to Jpeg dj Defer Javascript hw Flushes html io In-place optimize for browser idp Insert DNS Prefetch js Jpeg Subsampling pr Prioritize Critical Css rj Recompress Jpeg rp Recompress Png rw Recompress Webp rc Remove Comments cf Rewrite Css jm Rewrite External Javascript jj Rewrite Inline Javascript cp Strip Image Color Profiles md Strip Image Meta Data

Options CssInlineMaxBytes (ci) 10240 EnableCachePurge (euci) True EnableRewriting (e) 1 FileCacheCleanIntervalMs (afcci) 3600000 FileCachePath (afcp) /var/cache/mod_pagespeed/ FileCacheSizeKb (afc) 4096000 ImplicitCacheTtlMs (ict) 15549367000 LoadFromFileCacheTtlMs (lfct) 15549367000 LogDir (ald) /var/log/pagespeed LRUCacheByteLimit (alcb) 16384 LRUCacheKbPerProcess (alcp) 1024 MemcachedServers (ams) localhost:11211 RewriteLevel (l) Optimize For Bandwidth SslCertDirectory (assld) /etc/ssl/certs StatisticsLogging (asle) True