Add Robots.txt to prevent new site from showing up on google before it goes live.

SchoolofComputerScience / scs-issues

cs-dev issues repository.

http://cs-dev.cmu.edu

0 stars 2 forks source link

Add Robots.txt to prevent new site from showing up on google before it goes live. #66

Closed wesruv closed 7 years ago

wesruv commented 7 years ago

The new site is showing up in search results, to avoid any dings over duplicate content or confusing users I think we should hide it for now.

clizzy12 commented 7 years ago

Hey @wesruv, this actually is already set-up on our server. We use a node module called express-robots which auto-generates the file in middleware (https://github.com/SchoolofComputerScience/scs-app/blob/master/server.js#L58).

If you want to learn more about it, look here: https://www.npmjs.com/package/express-robots

wesruv commented 7 years ago

Excellent! I double checked and I have no idea how Susie saw it in search results... very strange. I'm not able to get google to bring up any pages.

Thanks!

RogueCSstaffer commented 7 years ago

I tried to search when I saw this issue and I couldn't find it either but was told that at some point it was out on Reddit as well. Haven't found evidence of that, however.