search

extension / learn

learn.extension.org version2; retired spring 2020
http://learn.extension.org
Other
3 stars 0 forks source link

robots.txt #60

Closed sdnall closed 12 years ago

sdnall commented 12 years ago

We need to determine which learn.extension.org urls wil be spiderable

sdnall commented 12 years ago

only event pages, /, and feeds/events should be spiderable. authentication rules will be put into place in the future, but it's probably a good idea to do this anyways