search

srushti / goldberg

Goldberg is a lightweight CI server written in Ruby which worries about Bundler & RVM so that you don't have to.
Other
243 stars 29 forks source link

Add robots.txt #114

Closed jasim closed 12 years ago

jasim commented 12 years ago

Search engine spiders are consuming an enormous amount of bandwidth trying to spider every build status. Only the home page and builds list (if required) should be allowed for crawling.

pahnin commented 12 years ago

Hi, I fixed this, I suppose. Here's is my commit message https://github.com/pahnin/goldberg/commit/7bc8414cd201032c3ba283c7ea9d2e951797e98b and link to repo https://github.com/pahnin/goldberg

Regards