Sitemap is generating URLs which are blocked in robots.txt and set to not include

carloscontrerasurdaneta / googlesitemapgenerator

Automatically exported from code.google.com/p/googlesitemapgenerator

Apache License 2.0

0 stars 0 forks source link

Sitemap is generating URLs which are blocked in robots.txt and set to not include #157

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago

How can I stop this? It keeps generating URLs from my Cart Directory, when it 
is set not to.  The directory is also blocked in the robots.txt file. 

The following comes up in the search results, and the sitemap GSG is generating 
a url for it why?:

www.domainname.com/mobile_sitemap_ca68bd15_000.xml.

I'm using a Linux/Apache server.

Please help.  Thank you.

Original issue reported on code.google.com by seocalif...@gmail.com on 13 Apr 2011 at 5:46