Open sanderheilbron opened 9 years ago
When a Allow: / statement or blank line precedes a Disallow: /foo statement, the Disallow: /foo statement always returns true.
Allow: /
Disallow: /foo
true
Unfortunately this does not match how Google interprets a robots.txt file.
When a
Allow: /statement or blank line precedes aDisallow: /foostatement, theDisallow: /foostatement always returnstrue.Unfortunately this does not match how Google interprets a robots.txt file.