Code review request

[GoogleCodeExporter]

Purpose of code changes on this branch:
replace in header.php

<?php if(is_search()) { ?>
    <meta name="robots" content="noindex, nofollow" /> 
    <?php }?>

with

<?php if((is_home() && ($paged < 2 )) || is_single() || is_page() ||
is_category()){
    echo '<meta name="robots" content="index,follow" />';
} else {
    echo '<meta name="robots" content="noindex,follow" />';
}
    ?>

When reviewing my code changes, please focus on:

by using WordPress conditionnal tags, this code ensure that homepage,
posts, pages and category pages are indexed by search engines spiders, but
prevent all other pages (feeds, archives, search, etc) from being indexed
and create involuntary duplicate content.

After the review, I'll merge this branch into:
/trunk/library/header.php

Original issue reported on code.google.com by dannydam...@gmail.com on 27 Jan 2009 at 9:58

[GoogleCodeExporter]

Original comment by dannydam...@gmail.com on 27 Jan 2009 at 10:00

• Removed labels: Type-Enhancement

[GoogleCodeExporter]

Tested and it's working fine. Though I think that category archives should be
excluded from indexing. 

Results:
index page:  <meta name="robots" content="index,follow" />
individual post: <meta name="robots" content="index,follow" />
page: <meta name="robots" content="index,follow" />
category archives: <meta name="robots" content="index,follow" />
tag archives: <meta name="robots" content="noindex,follow" />
yearly archives: <meta name="robots" content="noindex,follow" />
monthly archives: <meta name="robots" content="noindex,follow" />
search results: <meta name="robots" content="noindex,follow" />
index with pages (after 2 pages) <meta name="robots" content="noindex,follow" />

Original comment by garinung...@gmail.com on 3 Feb 2009 at 1:24

[GoogleCodeExporter]

I think about it and you are right. Categories may be exlude from indexing. 
Change
the code i really simple. and search reasult pages may have noindex, nofollow.

Original comment by dannydam...@gmail.com on 3 Feb 2009 at 11:38

[GoogleCodeExporter]

Suggestion for improving how pages are indexed:

<?php if((is_home() && ($paged < 2 )) || is_front_page() || is_single() || 
is_page()
|| is_attachment()){
    echo '<meta name="robots" content="index,follow" />';
} elseif (is_search()) {
    echo '<meta name="robots" content="noindex,nofollow" />';
} else {    
    echo '<meta name="robots" content="noindex,follow" />';
}
?>

This way, main index, front pages, individual posts, pages and attachments are
indexed; search results are excluded entirely from indexing while all other 
pages
(mainly archives - categories, tags, authors, date) are not indexed but links 
will be
followed.

Thoughts?

Original comment by garinung...@gmail.com on 4 Feb 2009 at 3:10

[GoogleCodeExporter]

Took the last offer ;-) .. REV 296

Original comment by chris.gossmann@googlemail.com on 4 Feb 2009 at 12:24

• Changed state: Fixed

[GoogleCodeExporter]

well done.

Original comment by dannydam...@gmail.com on 4 Feb 2009 at 2:19