See #25: Fix some common 404;

[kaihendry]

https://github.com/chernjie/hackerspace.sg/commit/28f023a4082f07ad94b2f96d131ea827b8750204#commitcomment-5226229

[notthetup]

Yea.. Why disallow robots??

Also why is rss redirected to the github rss?

[chernjie]

Look closely, robots disallow is at the top root (punch-hook). The one in template is allow all.

[chernjie]

Perhaps the RSS should be an events feed. I was thinking the closest to a list of activities would be Github's activity, which is why I added it. @notthetup would you prefer it to be gone too?

[kaihendry]

RE "RSS should be an events feed", no Ical should.

This pull request is nonsensical to me.

[chernjie]

Fix, happy now @kaihendry ?

[kaihendry]

I still think the robots.txt is un-needed / confusing. Is Googlebot hitting punch-hook a problem?

No

Do we need a robots.txt template?

No

[chernjie]

Either that or we remove punch-hook vhosts, what do you say? robots.txt in template is required as bots will always hit it. What's the harm in placing it there?

[kaihendry]

Why remove punch-hook from vhosts? That would break the Web hook.

What's your rationale?

robots.txt is not required. What's wrong with a 404 exactly?

Less is more.

[chernjie]

You do not need a brand new vhosts for a web hook.

Fine if you insist on not having a robots.txt file, just delete it.

[kaihendry]

It's neater having a domain instead of path which I think you are suggesting ?

[chernjie]

neater is subjective. neater in how you write index.cgi and not having to worry about the path issue, yes. But not neater in the perspective of a blackbox web service

[chernjie]

@laktek is there a way we let punch handle the generation of robots.txt? I think it would certainly be better.