Spiderable should return 404 when processing an unknown route

[jazeee]

Currently, all pages return with status code of 200, even if the route does not exist.

Returning 404 for non-existent pages is important for SEO.

This is a bit tricky in Meteor. Meteor and IronRouter don't really allow you to easily return 404's.

Even though IronRouter says it defaults to 404's, it doesn't seem to do that. Instead it renders a default template. I can't figure out how to make it 404. There are a number of discussions online...

• https://github.com/iron-meteor/iron-router/issues/237
• http://stackoverflow.com/questions/27001298/how-to-return-404-using-iron-router

What I was able to do is to specify a default route using

Router.configure({notFoundTemplate: 'pageNotFound'})

And for the pageNotFound template, I can add:

Template.pageNotFound.rendered = ->
    Meteor.isPageNotFound = true
    Meteor.errorResponseCode = 404

Then, I can use that in phantomJS. I have that working, which was a bit of a pain dealing with PhantomJS specifics...

[jazeee]

@dr-dimitru This should help address the 404 issue: https://github.com/jazeee/jazeee-meteor-spiderable/commit/3cd05c38afb6aa25da1e9d64242efd8dc1a370a2

Let me know what you think.

[dr-dimitru]

@jazeee Okay, you did exactly what I've missing all this day. I'll add part of your code into main What I currently did:

• Create server route which returns real 404 error, for example: /not-found
• On notFoundTemplate render if ___isRunningPhantomJS___ redirect to that real 404
• Prepend real response code to output
• In package phrase output to get real response code
• Cache with response code
• Return output with real response code

It will take around 1-2 days to test and done everything. What do you think?

[jazeee]

I made some minor corrections to clean up the server code. In particular, I am now storing the responseCode, instead of error, in the cache.

[dr-dimitru]

I believe we've solved this issue at last release