Support asynchronous server rendering (waiting for data before rendering)

[fdecampredon]

It would seriously ease the process of building something isomorphic if componentWillMount could return a promise and that react would delay rendering until that promise is resolved. I have seen attempt of doing something like that in react-router and rrouter, however giving this responsibility to each component instead of a router module would make more sense for me.

[sophiebits]

The main reason (I believe) that this doesn't exist already is that on the client side, you basically always want to show some sort of loading indicator instead of deferring rendering. (It would also make the code significantly more complex, but we can probably deal with that.)

[fdecampredon]

There is 2 cases that I find hard to solve without that:

• On the server side, if you want to fetch data before rendering you can't delegate data retrieval to components since you don't have the information of which component will be rendered
• On the client side the first time you mount your application after receiving pre-rendered html, even if you have some sort of cache from data retrieved on the server, you might want to use async method to retrieve those data, and that would prevent react from reusing the html.

react-async solve those problems with fibers, and cache. That do the trick but in my point of view those are just hackish solutions to solve a problem that can only be solved in core.

[syranide]

Color me uninformed on this subject, @fdecampredon say that componentWillMount is async and you don't return anything immediately, what is React supposed to render until there is, nothing? If so, why not just return nothing in render if there is no data yet? (Yeah I get server-side) Also, what should happen if props change before componentWillMount fires?

Personally, it seems like it's wrong to dispatch async requests during componentWillMount unless the component really is an isolated black box and also implements a loading indicator. As far as I understand it, React components should not be mistaken for more conventional OOP instances. In the best case, a React component is tool for visualizing the data in props, if it's interactive then possibly also state. It's a view, not a view and model.

To my ears, that sounds like the problem, React components shouldn't be the ones dispatching the async requests, you fetch all the data and when that data is ready, only then do you call React.renderComponent. Same solution client-side and server-side. You also get the ability to abort with an outcome of your choice if any async request fail.

Feel free to dismiss me if I misunderstood something, but it seems that you're treating React components as view and model, when (it seems) they're meant as just the view.

[fdecampredon]

Color me uninformed on this subject, @fdecampredon say that componentWillMount is async and you don't return anything immediately, what is React supposed to render until there is, nothing? If so, why not just return nothing in render if there is no data yet? (Yeah I get server-side) Also, what should happen if props change before componentWillMount fires?

I must admit that I did not think about all the cases ^^. This feature would be useful only the first time we mount the top level component, and on the server, it's true that otherwise In most cast you would want to display a loader indicator.

Personally, it seems like it's wrong to dispatch async requests during componentWillMount unless the component really is an isolated black box and also implements a loading indicator. As far as I understand it, React components should not be mistaken for more conventional OOP instances. In the best case, a React component is tool for visualizing the data in props, if it's interactive then possibly also state. It's a view, not a view and model.

In a way or in other you'll want that a 'top-level' component would be able to retrieve data, like it's done in the Flux sample. In this sample things are pretty simple because retrieving the list of todo is a synchronous operation, if it was not, and in case of pre-rendering on the server, we would render a first time with no data (and loose the pre-rendered markup from server).

In the case of a simple application with one set of data displayed by one view hierarchy there is still not so much problem, you can preload data and still keep the synchronous property of your store. Now in case of an application composed of multiple modules that you reuse across your application I would like to be able to treat those modules as separate applications that are able to 'subscribe' on different stores (that would be responsible for fetching data).

Perhaps That I get the things the wrong way but some discussion/sample around the web make me think there is something missing somewhere:

• In some sample it seems to me that @petehunt tried to achieve something similar.
• react-nested-router promote some similar mechanism in willTransitionTo, and some discussions make me feel that nobody has come with a proper solution.
• RRouter also provides some kind of mechanism to prefetch data as the component is being rendered/mounted.
• and finally react-async like I said earlier.

[mjackson]

@fdecampredon To be clear, the purpose of willTransitionTo in react-nested-router is not for loading data, specifically because returning a promise from that method will indeed block rendering new UI, which you don't want to do unless you absolutely have to.

[syranide]

@fdecampredon Everyone is still trying to figure things out, so it wouldn't surprise me if no one has a definitive answer. But I'm guessing the devs at Facebook must've run into this themselves a few times.

[tobice]

Any update on this? I've just started exploring React and I immediately run into this. Many people recommend React as a go to solution for building isomorphic apps, but as long as this is not solved I think it simply can't get the job done.

To my ears, that sounds like the problem, React components shouldn't be the ones dispatching the async requests, you fetch all the data and when that data is ready, only then do you call React.renderComponent. Same solution client-side and server-side. You also get the ability to abort with an outcome of your choice if any async request fail.

Feel free to dismiss me if I misunderstood something, but it seems that you're treating React components as view and model, when (it seems) they're meant as just the view.

If this is true then React is nothing more than a slightly different templating solution / view layer. And that would be a shame because there is such a potential. I really understand @fdecampredon when he mentions those complex applications composed of multiple modules. React would be perfect for this.

I don't think that this approach would mean treating a component as view and model. If you look at the Flux architecture, they think of React components as of controller-views that not only display data (from stores) but also invoke actions based on user interactions. And the actions then update the stores (= model). To me it sounds just like an obvious MVC architecture.

The problem is with the initial action that populates the store. It's fairly simple on the client side where the action can be invoked from the componentDidMount() method as recommended. On the server side on the other hand, we really need some special place and some kind mechanism that would delay the rendering until the action is finished and the stores are populated.

At this moment, it seems to me that the only way is React-async/Fibers + Flux. The nice thing about Flux is that we don't need some artificial cache to transport component states from the server to the client (like it's done in the original react-async example), we can simply initialize the stores and then send them to the client with the html markup (see this example).

But this solution is indeed hackish.

[mridgway]

I don't think it necessarily needs to be componentWillMount that is async; I'm not even sure it needs to be a lifecycle event. The real issue is that on the server side there is no way to analyze the component tree until after everything has been rendered to a string.

My ideal solution to solve this would be to allow "rendering" that would just build the component tree, then we would be able to traverse the tree to find components that need additional data, allow us to asynchronously load more data and "re-render" that components subtree, and then once we are ready for flushing the markup, allow us to convert that tree to a string.

This replicates what we're able to do in the browser: have a virtual DOM that we can re-render as we want. The difference is that in the browser DOM updates can be implicit. On the server we need to be explicit about when we render to a string so that we can perform updates to the virtual DOM based on async data.

[koistya]

My ideal solution to solve this would be to allow "rendering" that would just build the component tree, then we would be able to traverse the tree to find components that need additional data, allow us to asynchronously load more data and "re-render" that components subtree, and then once we are ready for flushing the markup, allow us to convert that tree to a string. — @mridgway

Yep, that would be good. Currently, rendering component twice on a server-side can be used as a workaround.

[mridgway]

I want to reference react-nexus as an example of what I'd want to be supported within React. It's essentially a rewrite of how mountComponent works except that it builds out the component tree without actually mounting it to the DOM or writing out a string. This allows you to traverse the component tree and fire off asynchronous methods while the tree is being built up. The issue with this implementation as it is, is that it throws away that first tree and then calls React.renderToString anyway, whereas it would be nice to take that pre-render tree and render/mount it.

I'm willing to work on this within core, but would need some pointers. I think one of the requirements is making ReactContext not globally referenced so that async wouldn't cause issues. Are there other globals that might run into issues with this as well?

[gaearon]

@mridgway If I'm not mistaken global ReactContext might be a compat thing and will go away in 0.14. But I might be wrong.

[mridgway]

@gaearon Yeah, that's the sense I got from the withContext deprecation.

[SpainTrain]

:+1: Using react-router for this ATM. @mridgway's approach sounds very reasonable.

[anatomic]

A slightly different take on this issue is how to handle asynchronous loading of code chunks produced by a bundler (such as webpack).

As discussed in this ticket - https://github.com/rackt/react-router/issues/1402 - the issue with async rendering support is the initial render appears to be null as the relevant chunks haven't yet loaded in the first render pass, resulting in a <noscript> at the root of the DOM and a checksum fail. Everything falls into place quickly afterwards, but it does result in a flicker in the UI when working locally and worse in the field, especially if the chunks to be downloaded are of a reasonable size (say > 30KB)

The ability to split up big apps into multiple chunks is important to us and having solved the data fetching issue (we used react-router and nested routes which can be traversed prior to rendering on the server to fetch data dependencies) this is the last piece of the puzzle blocking us from moving fully to a React solution for our front-end.

[syranide]

@anatomic That's not React's responsibility, it's your job to chunk appropriately and defer rendering until all necessary chunks has been loaded. To put it differently, if one of your components has a dependency on some external library, it's obviously your problem to satisify before trying to use it, React couldn't do it even if it tried, so the same applies across the board.

Feel free to implement alternative strategies that may suit you better, say <WaitFor for={MyAsyncLoadedCompSignal} until={...} then={() => <MyAsyncLoadedComp ... />} />. But these are inherently opinionated and not something React should or even needs to offer, so it's best left to the community.

[koistya]

It's better to keep async stuff outside the scope of React components, here is an example:

import React from 'react';
import Layout from './components/Layout';
import NotFoundPage from './components/NotFoundPage';
import ErrorPage from './components/ErrorPage';

const routes = {

  '/': () => new Promise(resolve => {
    require(['./components/HomePage'], HomePage => { // Webpack's script loader
      resolve(<Layout><HomePage /></Layout>);
    });
  }),

  '/about': () => new Promise(resolve => {
    require(['./components/AboutPage'], AboutPage => { // Webpack's script loader
      resolve(<Layout><AboutPage /></Layout>);
    });
  })

};

const container = document.getElementById('app');

async function render() {
  try {
    const path = window.location.hash.substr(1) || '/';
    const route = routes[path];
    const component = route ? await route() : <NotFoundPage />;
    React.render(component, container);
  } catch (err) {
    React.render(<ErrorPage error={err} />, container);
  }
}

window.addEventListener('hashchange', () => render());
render();

See Webpack Code Splitting, React.js Routing from Scratch and react-routing (coming soon)

[anatomic]

@syranide I'll keep working away at it, but I don't think it's as binary as you've put it above. We are using react-router so that may introduce some issues to the mix as the router is a component rather than sitting outside the React environment.

If we did the <WaitFor ... /> approach, surely we'll still get a different DOM on the first render which will still cause the flicker/disappearance of content?

[syranide]

If we did the <WaitFor ... /> approach, surely we'll still get a different DOM on the first render which will still cause the flicker/disappearance of content?

If you don't want flicker (some do) it's simply a matter of waiting for all the chunks you depend on to have loaded before rendering, webpack provides this out-of-the-box with require.resolve.

PS. Yes, react-router and whatever else you're using surely complicates the solution, but it's still not React's problem to solve.

[anatomic]

If you don't want flicker (some do) it's simply a matter of waiting for all the chunks you depend on to have loaded before rendering, webpack provides this out-of-the-box with require.resolve.

I'll look into that, my understanding of require.resolve was that it was a synchronous lookup of a module's id and didn't involve a trip to the server? We're using require.ensure to manage our chunk loading.

Looking at our code again I think we can circumnavigate the problem by making react-router think it's rendering on the server but then following the standard client-side approach:

const history = new BrowserHistory();

if (typeof history.setup === "function") {
    history.setup();
}

Router.run(routes, history.location, (err, initialState, transition) => {
    React.render(<Provider redux={redux}>{() => <Router key="ta-app" history={history} children={routes} />}</Provider>, document.getElementById('ta-app'));
});

[syranide]

I'll look into that, my understanding of require.resolve was that it was a synchronous lookup of a module's id and didn't involve a trip to the server? We're using require.ensure to manage our chunk loading.

Sorry, yes I meant require.ensure. The callback is executed only when all the dependencies are satisfied, so it's a matter of just putting render/setState inside of it.

[anatomic]

Ok, that's kind of how we were doing it, thanks for your replies. This feels like something that needs to be addressed in react-router so I'll carry on the discussion there - sorry if this has been the wrong place to have this conversation!

You are right that the require.ensure is the way to go, I guess our ultimate issue was that it should be linked to the route currently being visited so is directly tied to the router. With react-router being component based that ties it to the render tree. Without my hack above we're left fighting a way to view the tree before everything is loaded async (or duplicating the routing logic to allow for relevant chunks to be loaded at the top level).

[syranide]

You are right that the require.ensure is the way to go, I guess our ultimate issue was that it should be linked to the route currently being visited so is directly tied to the router. With react-router being component based that ties it to the render tree. Without my hack above we're left fighting a way to view the tree before everything is loaded async (or duplicating the routing logic to allow for relevant chunks to be loaded at the top level).

I'm not intimately familiar with react-router, but I still imagine it simply being a case of setRoute(...) => require.ensure([], function() { require(...); setRoute(...); }) which really isn't practical so you introduce another level of indirection. A map of routes and the async require.ensure loader for each. Write a helper function asyncSetRoute(...) { loadRoute(route, function() { setRoute(...); }}, now you call asyncSetRoute instead and it will defer updating the router until everything is ready.

Pseudo-code and kind of generic, but that seems like the overall approach to me. Maybe react-router should provide this, maybe not... maybe it's ideally provided as an external helper, I'm not sure.

[NickStefan]

So after hours of research, I'm just now confirming that server side rendering is _impossible_ unless you feed everything from the top down (?).

Possible short term solutions: A. Pre-fill a store and make server side loading synchronous

B. Feed everything from the top as one data input after async fetching your one data object.

re: 'A'. This wont work unless you already know what your render structure will look like. I need to render it in order to know my various collections/models/api dependencies. Also, how do I have the fetching be async on the client, but sync on the server, without having two different APIs?

re: 'B'. Basically the same problem as above. Are people having to make little dependency JSONs to go with each of their routes?

Are there any other short term solutions while we wait for a React supported solution? Any user land forks or plugins? https://www.npmjs.com/package/react-async ?

[gaearon]

@NickStefan

I don't understand the problem. :-(

1. Use Flux or Flux-like container (Alt, Redux, Flummox, etc) where stores aren't singletons.
2. Define static methods like fetchData on your route handlers that return promises.
3. On server, match route to components, grab fetchData from them and wait for them to complete before rendering. This will prefill your Flux or Redux store instance. Note that store instance is not a singleton—it's bound to a particular request, so requests stay isolated.
4. When the promises are ready, render synchronously with the store instance you just prefilled.

Here's a good tutorial for Redux describing this approach: https://medium.com/@bananaoomarang/handcrafting-an-isomorphic-redux-application-with-love-40ada4468af4

[NickStefan]

@gaearon I apologize if I confused you. Thank you for response. From your list, it sounds like I am correct in assuming the solution to server data dependency is to only ever define data needs at your root component (static methods / the article you linked to). If your data dependencies are defined at the root, it's much easier to pre-fill stores etc.

This is good flux practice, but isn't it potentially limiting? If I add a small component at the bottom of your view tree that needs some very different data, I then need to go edit the data dependencies at the root.

What I'm asking for is a way for deeply nested components to define asynchronous data needs.

If I have to add their needs to the root component, aren't I coupling the root to that one sub components needs?

[koistya]

@NickStefan with react-routing, for example, the async data fetching looks like this:

import Router from 'react-routing/lib/Router';
import http from './core/http';

const router = new Router(on => {
  on('/products', async () => <ProductList />);
  on('/products/:id', async (state) => {
    const data = await http.get(`/api/products/${state.params.id`);
    return data && <Product {...data} />;
  });
});

await router.dispatch('/products/123', component => React.render(component, document.body));

[tobice]

Those solutions work but only because all data pre-fetching is bound to the router. That is not a problem in most cases (it makes sense, URL defines what should be on the page and therefore what data is needed) but in general it's a limitation. You will never be able to create a standalone reusable component (i. e. Twitter stream, comments, calendar) that would handle everything by itself and all you had to do was just insert it into component hierarchy. The only way I've come across that makes this possible is react-async but that's pretty much a hack.

I suppose that React components are simply not intended to be used this way, they are still more views than controller-views. Probably a completely new library will have to emerge on the foundation of React.

My dream is that one day we'll be able to write a complete CMS using React, something like Wordpress, Drupal or Modx. But instead of HTML/PHP snippets we'll be composing the website from React components.

[gaearon]

@NickStefan

From your list, it sounds like I am correct in assuming the solution to server data dependency is to only ever define data needs at your root component (static methods / the article you linked to). If your data dependencies are defined at the root, it's much easier to pre-fill stores etc.

If I have to add their needs to the root component, aren't I coupling the root to that one sub components needs?

Not exactly at the root—at route handler level. There may be many route handlers in your app. Moreover, libraries like React Router support nested route handlers. This means your nested route handler can define a dependency, and the router match will contain an array of matched hierarchical route handlers, so you can Promise.all them. Does this make sense?

It's not as granular as “every component can declare a dependency” but I've found that in most cases limiting data fetching to route handlers (top and nested ones) is all I want. I have pure components below that accept data via props so they don't even know it's being fetched. It makes sense that they can't request it.

The “every component can declare a dependency” approach isn't practical unless you have some sort of request batching solution. Imagine if <User> declares that it needs an API call, and now you render a list of 100 <User>s—would you want to wait for 100 requests? This kind of data fetching requirement granularity only works when you have a request batching solution. If that suits you, Relay is exactly that. But you'd need a special backend for it.

[sophiebits]

@NickStefan We don't currently support per-component async fetching. We'd like to add it. This issue tracks our progress, though no one is actively working on it and it will take major restructuring so it will be a while.

[NickStefan]

@gaearon

The “every component can declare a dependency” approach isn't practical unless you have some sort of request batching solution.

After some more thought on this, I agree with you. What I was seeking isn't actually practical.

I originally had the thought that even your 100 example would be okay with team discipline (ie just make a <UsersContainer> that does 1 request for all 100). But that's not "people-scalable" to merely have a convention. Its probably for the best to enforce all the dependencies to the root or the router. Even Relay sort of forces you to declare dependencies at the Root with its three different containers (RelayContainer, RelayRootContainer, RelayRouter etc). It kind of proves the point that the only way is to essentially "hoist" your dependency declarations up the tree.

[sars]

Hi there! Is there any updates on this?

[vladkolotvin]

+1

[NickStefan]

I maintain that if you really think about this, you don't want to do this. I originally thought I wanted React to do async rendering. But others in this thread have convinced me otherwise.

Even in Relay, you basically have to "hoist" your dependencies up the tree with the root container declarations, relay containers etc. Synchronous rendering is the way to go.

Asynchronous rendering can be a nightmare. I speak from experience from working on a company codebase with backbone that was hacked to do individual updates in request animation frames.

[d4goxn]

Agreed. I used to think that we needed this, but it is actually backwards to have the view specify what data to load. The view is a function of the application state, which is itself a function of the request (and possibly the user's state, if there is a session). This is what React is all about; allowing components to specify what data should be loaded goes against the "one way data flow" idea, IMO.

[jimfb]

it is actually backwards to have the view specify what data to load.

I'm not entirely sure this is true. Sometimes, the component is the only thing that knows what data to load. For instance, suppose you have an expandable tree view that allows a user to browse a massive graph of nodes - it is impossible to know in advance what data needs to be loaded; only the component can figure that out.

Regardless, this discussion might become much more relevant if we pursue the idea of running React code within a web worker (#3092), where async is required to communicate across the bridge.

[d4goxn]

In the example of a large tree view, if I really wanted to be able to render it with a path already open on the server side, then I would add that path to the URL structure. If there were several complex components like this, then I would represent their states with GET parameters. This way, those components get all of the benefits of SSR: they are crawlable, can be navigated using history, users can share links to a node within them, etc. Now, we also have a way for the server to determine what data needs to be fetched in order to render a response.

update My bad mistaking a graph for a tree, but I still feel that the state of a user's view of that graph should be represented by URL structure. Also, didn't realize that you actually work on React. If you are working on some framework for integrating a data layer with the view isomorphically, that's awesome. But I think we can agree that that goes beyond the domain of the view, and that React should not take on the role of a full stack controller.

[olalonde]

Didn't read the whole thread so sorry if this was discussed already.

One thing that would really help would be if there was a react-dom/server method that would just "start/instantiate" a component and fire lifecycle methods but let the developer decide when he is ready to render the component to an html string. For example, the developer could use setTimeout or something more reliable to wait for async methods to complete.

This is the "hack" I'm using with redux at the moment to achieve this:

  // start/instantiate component
  // fires componentWillMount methods which fetch async data
  ReactDOM.renderToString(rootEle)

  // all my async methods increment a `wait` counter 
  // and decrement it when they resolve
  const unsubscribe = store.subscribe(() => {
    const state = store.getState()
    // as a result, when there are no more pending promises, the wait counter is 0
    if (state.wait === 0) {
      unsubscribe()
      // all the data is now in our redux store
      // so we can render the element synchronously
      const html = ReactDOM.renderToString(rootEle)
      res.send(html)
    }
  })

The problem with this approach is that the second ReactDOM.renderToString fires all the componentWillMount methods again which results in unnecessary data fetching.

[bruno12mota]

Hey guys! is this something currently being worked on? I think this issue is growing in importance. I've recently made a library that sort of extends react redux's connect to a dataConnect where you can also specify the container data requirements. These data requirements are then bundled runtime and queried with GraphQL in a single request. I think this is the future of data specifications as it promotes composability and isolation and also ensures a more performant fetching. (all concepts seen on Relay)

The problem with the above is server side rendering. Since I can't statically analyze which data requirements I'll end up with, currently, I'd need to render one time to get the bundle to query from the DB, hidrate the redux store and then re render to get the final string. The issue is similar to @olalonde's one.

Would be fantastic to able to trigger actions and updates to the react elements tree and get the DOM string result on demand. Here's how I picture it:

const virtualTree = ReactDOM.renderVirtual(rootEle);

// get the bundled query from redux store for example
const bundle = store.getState().bundle;

// Fetch the data according to the bundle
const data = fetchDataSomehow(bundle);

// hydrate store
store.dispatch({type: 'hydrate', data});
// components that should update should be marked on the virtual tree as 'dirty'

virtualTree.update(); // this would only update the components that needed update

const domString = virtualTree.renderToString(); // final result

Other option is to simply let it update at will like it would be on the client side, having all the hooks present and working such as didMount. But instead of mutating DOM it would just mutate the virtual dom representation, and render to string on demand as well.

What do you guys think? Something to consider or I'm seeing it completely wrong?

[d4goxn]

Hi all. I've been subscribed to this issue for a year or so now. Back then, I thought that I needed to be able to specify the data that should be loaded from within components, because components were my primary unit of modularity. This issue was really important to me, and I thought that for sure it would get resolved in the next version of React.

Since then, I have developed a much deeper respect for the ideals behind REST/HATEOS, because of the large scale simplicity that emerges in a system of applications (browsers, crawlers, application servers, proxies, CDNs, etc) when its guiding principles are followed. As it relates to this issue, the URL should be the one true representation of the application state. It, and only it, should determine what information is required to service a request. The view layer, React, should not be determining this; it should be constructed based on the data. The view is a function of the data, and the data is a function of the URL.

I have hesitated to throw this out there in the past, because I keep hearing that this is a nice idea, but that the real world is just too complex for this to work. And occasionally, I hear of examples that make me take a step back, and wonder if I am being too pedantic, or too idealistic. But as I mull over these examples, I inevitably find a reasonable way to represent the application state in the URL.

• Does your state have independently varying parameters? Represent them as separate query parameters.
• Is your state (or part of it) too big to fit into a URL? Name it and store it in an immutable datastore. Refer to it by name / ID.
• Is your state changing so fast that you just can't store all of it forever? Congrats, you have legit big data, spring for storage and start thinking about clever stuff to do with it. Or, you can shy away, and and just mutate your data with UPDATE queries, and accept that you cannot forever-cache all the things later (*).
• Do you have views that are different for different users, but served at the same URL, e.g. a personalized home page? Navigate upon user identification to a URL that includes the user's ID.
• Are you building over an older application, where you do not have the option to break the old URL structure? That's painful, I'm in the same boat. Redirect the old URL structure to a well architected URL structure, translating session data (or whatever) to URL path segments and parameters.
• Do you have little or no control over the architecture of your application? That is not the case that React was designed for, and we don't want React getting mangled to make it fit into something like a mutually compatible Wordpress / Magnolia / Umbraco architecture.

What do you get for all of that work, that you would not have otherwise had?

• The ability for one user to bring another user to the same place as them in the application, by sharing a URL.
• The ability to navigate the application using all of the tools that the browser provides. The standard client is in its element.
• The ability to offer complex pagination in a way that is simple for the client to follow: a next link in a response. The FB graph API is a great example of this.
• A detailed graph of your user's workflows in Google Analytics.
• The option of constructing said graph yourself from nothing but request logs.
• An escape from the star-route of chaos: Instead of matching all requests as an app.get("*", theOneTrueClientonaserverEntryPoint), you get to use your server application framework as it was intended. You can return correct HTTP status codes from requests, instead of 200 OK\n\n{status: "error"}. The star route is enticing, but it leads to madness.

So, now that React, our star tool, isn't controlling the operation, how do we get our data? How do we know what components to render?

1. Route to a data access function and fetch data for the relevant request parameters. Repeat until you have parsed the whole URL and have a complete context object.
2. Transform the context into the most simple response object possible, as if you were going respond to an API request. Are you serving an API request? Then you're done and DRY.
3. Pass that minimally complex, but possibly large, object to the top level component. From here, it's React component composition all the way down.
4. Render to string, respond. Let your CDN know that it can cache this forever (* unless you sacrificed this option above), because CDN identify by URLs, and all of your states have a URL. Sure, CDNs don't have infinite storage, but they do prioritize.

I don't mean to troll here, but I do feel strongly that the core theme of the requests in this issue are misguided, and that React should not implement anything to accommodate it, at least not for the reasons that I have seen here over the past year. Some aspects of good application architecture are not obvious, and web is especially hard to get right. We should learn from and respect the wisdom of our ancestors, who built the Internet and the Web, instead of sacrificing elegance for the sake of comfort.

That is what makes React great: It is the view. The best view. Only the view. λ

[bruno12mota]

Will have to disagree with you @d4goxn. I'm not trying to make React more than a view layer, and routing is important but definitely not enough to specify all data requirements. By specifying data requirements at a component/container level you're not making React more than a view layer. This just provides you a much better isolation and workflow for your app. It's not just a simplification of the work but a need for a big application. Imagine my app has 30 routes and I want to add a user profile component on each one. Following a route alternative where all the data requirements are specified for each, you'd have to go through the 30 routes to add that data dependency. When with if you specify your data needs in a container for that component, you just have to add the component where you want. It's plug and play. Not only this but the query for all components data dependencies can be optimized. Relay is a good example of this, and the talks about it explains this much better than me.

I have much respect for old wisdom, but that shouldn't be a limit for evolution and creating new standards, that's the way I see it at least.

My proposal is basically to not change React, but have a 'virtual only' way of altering the dom/components tree, basically a React that can be 'run' on server side, which I think is pretty simple to do (just block actions to alter the DOM). You can still have caching and have a CDN in place. With the growing dynamics of sites and applications nowadays static caching will tend to reduce but that's another topic 😄

[d4goxn]

If the view specifies dependencies on data, then you will need some way of optimizing the dependency graph and transforming it into a minimal number of queries; you cannot prepare the data before constructing the view, although you could split it into two phases, which is what I understand the plan to be in this thread. Take a collection of moderately complex components that would each have a query of their own, for example; perhaps they are not all instances of the same component, and there isn't a pattern that could be collapsed into a single query. I suppose this is what GraphQL is tackling. You will still need to implement or integrate GQL servers for each of your datastores. Sorting out which parts of the optimized GQL query should be handled by which datastore sounds pretty complex, but then my proposition has some of that complexity too.

In the example, where there are a large number of routes all needing the same data, I actually would not see that as a reason to exclude the identifier for that data source from the URL. I would mount a small data fetching middleware module fairly close to the root of the stack, which would attach that user object to a context and then pass the context along to more middleware, on its way to an end route handler. The root React component might not care about that particular part of the context, but it would pass it to the next level of children who might about it, or have descendants of their own that care. If this does introduce unreasonably complex or deep wiring, then something like a Flux store might be called for, but that is a big topic on its own.

Separation and isolation: I feel your pain there. In my proposal, we have two very different systems, transforming data from a form that is optimized for storage and retrieval, to a form that is optimized for abstract simplicity. Then my view is transforming that into a form that suits it, as it is passed down the component hierarchy. Keeping those two systems loosely coupled while adding a feature that requires additions to both systems is not what I would call hard, but it is also not the path of least resistance. The mental switch between programming for a data store and programming for a dynamic UI is real. We used to have separate backend and frontend developers, and HTTP served as the interface between these two paradigms. Now I am trying to internalize it into the one application, and you are trying delegate it.

I am not convinced that the increasing complexity and activity within applications precludes the client state from being represented by a very small object, that references a much larger data set on the server. Consider a massive multiplayer first person shooter: there is a lot happening fast, and you want to transmit the minimum amount of information necessary from the client to the game host/server. How small could you get that? A map of all input states; a timestamp + uncertainty range, a ~110 bit field for the keyboard, and a few dozen bytes for the mouse / joystick and VR headset orientation. The client would be predicting, optimistically rendering and physic-ing, and the server would be deriving a huge amount of information from the client's small state and state history, and returning a fairly large amount of data to correct and update all of the clients (all the same params for all nearby agents, asset pushes), but, that stream of client requests just might still fit comfortably into 2KB requests. And that might be a boon for the application architecture.

I won't ask you to educate me on Relay and GraphQL; I have only superficially explored them, before their APIs reached a stable version (are they locked down now?) If you are still convinced that using components to select GraphQL schemas, which specify data dependencies, which determines what actual data needs to be fetched, is a good plan, then maybe it is time I took another look at them. I have some hard questions about that architecture, but I'm about to go way off topic.

:beers:

PS I didn't mean to suggest that HTTP would be a good communication protocol for an MMOFPS

[bruno12mota]

@d4goxn I completely understand your hesitation and skepticism about this. I never thought about this until starting to work with GraphQL and later the introduction of Relay's concepts. I really recommend you to give it a look. And implementing GraphQL in a new or existing server is not as difficult as you might picture, even if you have multiple data stores. Not wanting to go off topic as well, but this is an important discussion.

I believe that this level of abstraction is the way to go. I've been using it on Relax CMS and the workflow and isolation is a bliss to work with. Not only that but the data requested is not more, nor less what the ui needs, and that's made automatically by collecting which data each component needs and merging it. This is done by Relate http://relax.github.io/relate/how-it-works.html if you're interested in checking. With this I can also include any component anywhere in my application and be sure that it will work as an independent block of my application.

The only thing still to figure out is the server side rendering. After going through react's source code I believe there might be already a solution as I described, if not I could work on a solution if someone from the react team agrees with this.

[tobice]

@d4goxn I will also dare to argue with you :) Your original post contains a statement that the view is a function of the data, and the data is a function of the URL. I wouldn't call this exactly revealing. It applies pretty much to any website or any web application which is not completely random (there is a question whether for example the state of open dialog windows should be also part of the URL). As we all had math at school, we know that function can be composed. So the actual statement might be that what you see on the screen is a function of the URL. Again, nothing really revealing, it's just a formalization of the obvious.

For me, the main question is how to construct such a function.

The approach you suggest feels very similar to the good old server-side MVC applications (e.g. Spring MVC is a good example). The current URL activates the corresponding controller method which does the business logic. It fetches all required data and passes them to the view. My problem with this is following: When you look at the generated web page, it's some kind of hierarchy of components (not necessarily React components). What you have to do is to create or generate the hierarchy from the URL. But you have to do it twice! First, you need to decode the hierarchy in the controller to know what data you need to fetch, and second you need to decode the hierarchy of the view to... well... render the actual view. I don't think it's a very DRY approach.

I'm not that familiar with Spring MVC but I use very often another MVC framework (PHP framework called Nette) which addresses this problem in a very similar way to how I would like to use React. This framework also supports components. The idea is that for each component (e.g. a form) you define a factory in your code which is responsible for instantiating the component and especially for loading all necessary data. Such a component is then possible to be included anywhere in the view. So as you write the HTML code and create the view hierarchy, you can simply insert a component and the underlying factory takes care of necessary initialization. The component behaves like a small independent controller, it has its own life cycle, it handles its dependencies and it can be even parametrized from the view which increases its re-usability.

I have been using this approach with React on client-side as well and it seems very viable. The React components have been from the beginning denoted as controller-views and that is how I use them. I have my controller components that are responsible for data fetching and then I have my view components that care only about the visuals. The whole page is composed from components of both types. They are nicely decoupled and easily re-usable.

My application is not isomorphic (or universal as they call it today) as I didn't need it but the stack I'm using should be capable of that (and I believe that apart from Relay which is still experimental, this stack walked the longest path in this matter). FYI, this is how it looks like:

• The core tool is react-router. It allows to hook asynchronous data request on each URL and it works both client-side and server-side. Formally speaking, this approach suffers from the problem I mentioned earlier. Those controller components are not possible. But what matters here is the design of react-router. It allows to define both the data-hierarchy and the view/component hierarchy in the same place and the actual route definitions are also hierarchical. It feels very React-like.
• The whole state (the data) is handled with redux. Besides all other advantages, it keeps the whole state in a single object which means its very simple and natural to create the object on server and send it to the client. Also thanks to the connect() approach, it's not necessary to pass all the data from the top level down using props (that would be absolutely crazy considering the current size of my application).
• Another piece of puzzle is reselect which allows us to keep the state as small as possible. And also greatly increases the decoupling factor.

It's still not perfect but it's a significant progress from what was available when I first came across this thread. Sample implementation can be found here.

[decodeman]

I too have almost all isomorphic working, except need a way to accomplish server side rendering when data fetches are at component level. Without adding to the philosophical argument right now, for a long time I have been successfully using vanilla react (plus homegrown routing) with components fetching their own data in componentDidMount. I really dislike the idea of maintaining a redundant structure to define my component and data dependencies - that is already defined in my component tree, and here I just somehow need perhaps multiple render passes that incorporate data as it is fetched asynchronously, until the full component tree has been resolved.

Right now, I just wanted to second the growing importance of this for isomophic react, imo. I will try to work out a solution in the meantime. Thanks.

[bruno12mota]

@decodeman in case you're interested, for Relate, I've done a minimal react traverser to get data dependencies from the components https://github.com/relax/relate/blob/master/lib/server/get-data-dependencies.js. This way I can get all the data dependencies -> fetch the data -> react render to string.

[SpainTrain]

@decodeman, FWIW we have been using the double render approach (like what you allude to) successfully in a large production app. It is inspired by this redux-saga example, but the general approach should work well with any type of async loading. You just need to move loading to componentWillMount. To help keep it manageable, we also have higher order components that take care of common loading scenarios (e.g., check for prop x and load if nil.

[fkrauthan]

I also think it would be very important to be able to load data within components and have some sort of lazy render method. Maybe to make both sides happy create a new method called asyncRenderToString and add a new "lifecycle" event called asyncOnLoad or something like that. This will only be called if you call a asyncRender method. In terms of code duplication between server and client side this can be fixed by developers of just moving common loading code into a separate method and call that method from asyncOnLoad and componentMount. Detecting if asyncOnLoad is done should probably use a returned Promise (if undefined is returned/method not defined it should directly call the applicable lifecycle events and then render) otherwise it waits until the promise resolves.

I think a solution like that would solve all the hacky ways we have to use at the moment for proper server side rendering. Including for example allow easy server side rendering of react applications with react router v4 (which not longer uses a centralized route config which is right now the only way to get isomorphic applications working)

[ghost]

This has been solved its called stream check out react stream.

On Thursday, November 3, 2016, Florian Krauthan notifications@github.com wrote:

I also think it would be very important to be able to load data within components and have some sort of lazy render method. Maybe to make both sides happy create a new method called asyncRenderToString and add a new "lifecycle" event called asyncOnLoad or something like that. This will only be called if you call a asyncRender method. In terms of code duplication between server and client side this can be fixed by developers of just moving common loading code into a separate method and call that method from asyncOnLoad and componentMount. Detecting if asyncOnLoad is done should probably use a returned Promise (if undefined is returned/method not defined it should directly call the applicable lifecycle events and then render) otherwise it waits until the promise resolves.

I think a solution like that would solve all the hacky ways we have to use at the moment for proper server side rendering. Including for example allow easy server side rendering of react applications with react router v4 (which not longer uses a centralized route config which is right now the only way to get isomorphic applications working)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/facebook/react/issues/1739#issuecomment-258198533, or mute the thread https://github.com/notifications/unsubscribe-auth/ATnWLUIEJw4m1Y3A4oGDOBzP6_ajDcqIks5q6g4_gaJpZM4CHAWq .

[fkrauthan]

@yamat124 if I search for react stream I only find this project: https://github.com/aickin/react-dom-stream

At least based on the documentation there is no way to have a lifecycle event for preloading data which delays the next render call for subchilds until the Promise is resolved. Or are you talking about something else?

[anaibol]

What about https://github.com/makeomatic/redux-connect?