SEO links without server-side rendering

[ingwet]

Hello everyone!

I am rather new to React, and currently finishing the project with it. I am using react-router to manage the routes, and the question that I've bumped into was making my content crawlable by google for SEO. As was advised in google docs, it's expected of ajax app to have links in the example.com/#!about (with #!) format to make google request static pages with escaped fragment from the back end. So you basically format your urls like that and make backend render static version to example.com?_escapedfragment=about and it should be indexed by google just fine.

However, the issue I've run into is that you can't make react-router format urls like that. So my question is "Is it a bug or a feature?" :)

What I've found out that even though react-router can't make your urls look that way (say with Link component), it works just fine parsing them. So basically if I click example.com/#!about link in my app, react router will redirect me to example.com/#/!about (notice one more / between # and !) and render my route !about.

So what I've descended to was rewriting the Link component and making it render the url google-way (example.com/#!about), specifying paths on my routes ( name="about" path="!about") and letting router ensure hashes in the path.

What I am wondering is that, maybe, there is a better way to do that (without server side rendering)?

Thanks for taking your time to read it

[agundermann]

It's intentionally not supported: #601

The way I understand the docs, you could also use pushState and include the fragment meta tag <meta name="fragment" content="!">, so google would request http://example.com/about?escaped_frament=

Is there any reason you don't do server-side rendering? Should be even better for SEO (and for users).

[ingwet]

@taurose, thanks for your answer. The reason behind not using server-side rendering was the problems of setting it up with our backend. We will definitely look into it closer at a later point.

With regard to the pushState, as far as I understand it's no good using it without server-side rendering. Opening a link to (http://example.com/about) [http://example.com/about] in a browser, the backend will need to give back the same page as if opening (http://example.com/) [http://example.com/] and let the react-router to the routing? (so that it will basically break the SEO altogether?)

If you've happened to come across the detailed explanation/tutorial about setting these things up, I would greatly appreciate a link in that direction :)

[agundermann]

With regard to the pushState, as far as I understand it's no good using it without server-side rendering. Opening a link to (http://example.com/about) [http://example.com/about] in a browser, the backend will need to give back the same page as if opening (http://example.com/) [http://example.com/] and let the react-router to the routing? (so that it will basically break the SEO altogether?)

Yes, you would need to send the same html for all paths. For real users, react-router would start, parse the URL and the content would be loaded. Googlebot would notice the meta fragment tag, ignore the current page, and then request and process the same url, but with ?escaped_fragment= appended, i.e. you'd have to check for the existence of that query parameter server-side and then send a pre-rendered version of the requested path. I don't see how this would break SEO any more than hashbangs.

I haven't actually worked with ajax server-side rendering before though, so perhaps you should wait for someone else to chime in.

Also, I didn't find many useful resources. There's a blog post about making an express middleware for pre-rendering (mean-seo) work with pushState instead of just hashbangs. Perhaps you might want to look at that module.

[ingwet]

thank you, @taurose. I will explore this topic. So the optimal option would be to check whether the user's browser supports pushState (search engine bots should I assume) and then giving them either nice urls (if yes) or with hashbangs (if they don't support it)?

It would also make sense then to parse the url first (e.g. if someone has shared a hasbanged version with a friend, who supports pushState or vice versa) and modifying it manually to fit the needs then?

[agundermann]

I would simply avoid using hashbangs altogether and only use HistoryLocation, which automatically falls back to RefreshLocation (reload from server). You can find a lot of discussions about the drawbacks of hashbangs on the internet.

[ingwet]

@taurose I've been working on transferring the whole app to server-side rendering today, and the issue I've ran into was inability to make server-side Router use HistoryLocation (to render proper links to user) and match the path that I am passing to it as a string (from my backend).

Is there any workaround for that? spent a while looking for it, but nothing :(

[agundermann]

For server-side rendering, you're supposed to pass the requested path (as a string) instead of any Location object to Router.run. Have a look at the snippet there: https://github.com/rackt/react-router/blob/master/docs/guides/server-rendering.md This will generate normal/nice URLs that work with pushstate and even without javascript. Transforming URLs would only be needed with HashLocation, so you shouldn't have to worry about that.

[ingwet]

Got it to work, but with php it's been a hell. The router wouldn't run if it was instantiated only once, so made it work with this - may be it'll be useful for someone else. I am closing this one.

renderPath = function(path) {
  var str;
  Router.run(AppRoutes, path, function (Handler) {
    str = React.renderToString(<Handler/>);
  });
  return str;
}
module.exports = renderPath;

[izziaraffaele]

@ingwet do you use a fake dom (jsdom) to pre-render your components server side with PHP? Because I'm trying to move my application on server side but as soon as I include react-router php-v8js complains that "document" is undefined (V8Js::compileString():23068: TypeError: Cannot read property 'documentElement' of undefined)

Could you please share with me an example of how you integrated the router with PHP server side rendering?

[izziaraffaele]

I realized that maybe is a problem of material-ui making use of modernizr so I opened an issue in their repo (https://github.com/callemall/material-ui/issues/1551). Sorry for the "spam". Anyways let me know if you have any suggestions regarding a general setup of a react app rendered server side by php. It's already 2 days that I'm struggling with this...

[ingwet]

@izziaraffaele no, I've tried jsdom, but couldn't get it working. The final solution was to have a node.js internal server running locally on our production machine, php asking it to render React to a string (via local port 3000), and then outputting this string to html.

Regarding the lack of document/window variables on the server side, we ran into this issue as well. For us what we did was to create a global object in javascript to mock them and avoid these errors

    var window = {};
    var document = {};

So at the end of the day we had two main.js files for initiating our code - one for frontend and one for backend. I can share it with you, if you'd like

[izziaraffaele]

@ingwet I just don't like the solution of having a node server rendering my react components. What you do to mock document and window I think is more or less what the guys of FB are doing here https://github.com/reactjs/react-php-v8js/blob/master/ReactJS.php#L55

But they use a very annoying approach to load JS code and execute it on PHP that doesn't fit my workflow. I just want the same JS code I had when I was working with react only in front-end, built with webpack as before, packed as I want (1 vendor fie and 1 app file), working if rendered by PHP or in the browser.

[ingwet]

@izziaraffaele I understand what you want, but I am afraid I haven't been able to make it work this way. If you find a nice solution, I would appreciate you sharing it)

[izziaraffaele]

@ingwet Actually I've got something more or less working in my case. I didn't got the routing working yet ( I just switched to https://github.com/STRML/react-router-component that actually worked straight forward ) but the rest of the setup is working without any node server. You can find it here https://github.com/izziaraffaele/reactavel.

Just to give you some references I use laravel as PHP framework and webpack to build my assets. I've my general App.jsx component that builds the general interface and render the "location" (that's how routes are called in the router I'm using )

I also have other 2 files, bootstrap.jsx and bootstrap-server.jsx, that are my entry points for webpack. You can find some example in the repo.

bootstrap.jsx just render the application as I usually do when I work just in the browser. bootstrap-server.jsx defines some mocks and polyfills for object not available server side and renders the app to a string ( for example I use Intl and here I define a polyfill for it )

The last important js file is vendor.js. It's built by webpack as well using the CommonsChunkPlugin and contains React.js + the rest of the thirdparty code used by the app

Then in PHP you can do something like

$errorHandler = new Reactavel\ErrorHandler($psrLogger) // you can replace this if you want
$reactavel = new \Reactavel\ReactJs('path/to/compiled/vendor.js',$errorHandler);
$appComponentProps = ['route'=>'/page/route'];
$appComponentString = $reactavel->addAppParams($appComponentProps)
            ->addAppSource('path/to/compiled/bootstrap-server.js')
            ->getMarkup();

// you can now print the $appComponentString into a <div id="app"></div> so that bootstrap.jsx can render the component on the browser correctly

I'll add some testing and example later on the repo. Now I'm very busy get my project done :D

Let me know if that was useful ;)

[ingwet]

It is quite interesting indeed, will give it a try when rewriting this part of the code) thanks!

[damhonglinh]

As announced in Google blog, Google now can crawl JavaScript-heavy one-page websites. So we don't have to concern this issue much any more (although the issue was closed a long time ago already).