Open sanderheilbron opened 9 years ago
When a Allow: / statement or blank line precedes a Disallow: /foo statement, the Disallow: /foo statement always returns true.
Allow: /
Disallow: /foo
true
Unfortunately this does not match how Google interprets a robots.txt file.
When a
Allow: /
statement or blank line precedes aDisallow: /foo
statement, theDisallow: /foo
statement always returnstrue
.Unfortunately this does not match how Google interprets a robots.txt file.