Closed GoogleCodeExporter closed 9 years ago
A robots.txt like
User-agent: *
Allow: /1.html
Disallow: /
works fine.
I'm not sure whether Disallow: * is just a wrong way of specifying pages that
are not allowed, but would expect it to work since Allow: * is a valid entry in
robots.txt.
Original comment by acrocraw...@gmail.com
on 25 Feb 2013 at 8:28
Disallow: * is not a valid syntax. It should be Disallow: /
The same for Allow. See
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449&from=3523
7&rd=1 for samples.
-Yasser
Original comment by ganjisaffar@gmail.com
on 2 Mar 2013 at 7:38
Original issue reported on code.google.com by
acrocraw...@gmail.com
on 22 Feb 2013 at 8:53