Router/History v2

[jamesplease]

Backbone's router is inspired by Ye Olde Routers from several years ago, before single-page applications were a 'thing,' and when one usually reached for a handful of jQuery plugins to build a JavaScript app.

These routers are distinguished by three characteristics:

1. Each route is independent from all other routes
2. Each route is a callback that is triggered when the associated URL is matched
3. The router is used to bootstrap the page, and is never used again

It's pretty clear that Backbone's router exhibits the first two characteristics. Over time, changes to the Router have made number 3 a li'l less obvious. The existence of the trigger option and the fact that it is false by default are evidence of that third characteristic. Numerous blog posts and books always warn against setting trigger: true. Additionally, @jashkenas expressed to me that this is how he believed the router should be used when we chatted about it at BBConf a few months ago.

Every other popular JavaScript library/framework has moved away from this pattern. In its place, a style of routing pioneered by Ember has become the de facto way to structure single page apps. Ember's router went on to heavily influence the React router, Angular's UI Router, and Angular v2's router.

While reaching feature parity with Ember's router might be a bit much for Backbone (it would prob. increase the size of the library by 2x lol), I think that looking there for inspiration and coming up with a minimal version of that sort of routing system would tremendously improve Backbone. And I think it could be done in a pretty small amount of code!

Further, by providing the right hooks, it would make it easier for third party developers to create an Ember-style router for Backbone as a third party library, which is quite the challenge right now (there's more on that at the end of this post).

These are what I've gathered to be a few of the important characteristics of those routers:

1. Routes are states, and the router is a state machine.
2. Transitions between states are asynchronous.
3. State can optionally be encoded into a URL when they are transitioned into.
4. States can have children states (this is where a large part of the complexity of Ember-style routing comes from)
5. Routes, at the minimum, have two callbacks associated with them: an asynchronous callback and a synchronous callback.
6. A route is activated each and every time the URL changes.
7. History is a read/write interface to the browser history, and has no other roles

FAQ for some of those points:

Why two hooks?

There are generally two things you want to do when the URL changes:

1. Fetch data (maybe)
2. Render a view (also maybe – but most of the time)

Generally, fetching data is asynchronous, and showing some HTML is synchronous. So, two hooks.

router.js breaks up the async method into three async methods to cover a variety of use cases. This complexity covers important use cases for transitions, but, again, it might be out-of-scope for BB to cover all of those cases. The cool part is that a single hook could be split out into 3 hooks by the plugin authors.

Why nested routes?

Nesting routes makes it easy to compose nested view states. It also aids with preserving view and data state.

Imagine a route called books. The URI for this route is also just books.

Then imagine two child routes: book (with URI :id) and authors (with URI authors).

If a user is at books/2, then books and book are both active. If the user then navigates over to books/author, we wouldn't want to re-fetch all of the data for books again! We've already got it! We also probably would not need to render the books view again.

This is just one edge case that the router solves for. As noted above, nested states is most likely far too complicated for Backbone to include. A flat route structure would keep the code small, but the right hooks would allow someone to re-write how routes transition.

A state machine? Well how come?

Because representing many applications as states that the router transitions between truly does simplify things so much. As an added bonus, the router can also be used to manage the state of apps that don't persist their state as a URL, like embedded widgets or whatevs, since encoding state into a URL is optional.

Why can't you create this this with the current router?

It is very difficult right now. There are two main reasons why:

1. The current router was developed with a different use case in mind, which makes the hooks it provides very bad for this purpose
2. The roles of history and the router are conflated, I think (to put it another way, backbone's router does not have characteristic 7 of Ember-style routing above)

Recent changes to the execute method have been an attempt at resolving the first problem, but they're sort of just band aid solutions over a more fundamental problem.

Examples of roles that History should not do (I think) include:

1. URL match algorithm History#loadUrl
2. Route execution History#loadUrl

I would argue that the router should manage both of these things, and they should be easily-overridable (and separate) hooks.

I'd be willing to put together a PoC router if there's any interest in something like this in Backbone.

thanx4reading

[jridgewell]

Examples of roles that History should not do (I think) include:

1. URL match algorithm History#loadUrl
2. Route execution History#loadUrl I would argue that the router should manage both of these things, and they should be easily-overridable (and separate) hooks.

I completely agree.

I would much rather an event system alerting Router of changes in the URL, with the router holding onto its own routes and detecting if any match. This would also help with #3440, allowing the router to trigger errors, etc.

[akre54]

Generally, fetching data is asynchronous, and showing some HTML is synchronous. So, two hooks.

Couldn't you do your async model stuff yourself in the synchronous callback, the way it is now? Most often if I'm re-rendering a route I already have the data I need. Two callbacks seems like overkill. Backbone doesn't need to be handling your fetching for you.

I've used Ember router before and found it massively overengineered. I don't think we need to go down that, ahem, route.

As an added bonus, the router can also be used to manage the state of apps that don't persist their state as a URL, like embedded widgets or whatevs

The URL is a pretty poor way to encode app state. It's much better to serialize to a cookie, localstorage, or the server, depending on your use case. Backbone doesn't need to handle this for you.

I think nested routes is another area where things get dicey pretty quickly. I agree it's a common pattern to nest resources, but in general you don't want to go more than one or two levels deep. I'm not sure we need to be in the business of wiring up connections between disparate parts of your app. And given the complications, it seems best if the book route handles both books/:book_id/comments and books/:book_id/comments/:comment_id, which it does pretty handily already with the optional () syntax.

But I agree with you that History probably shouldn't be responsible for handling your route callbacks. That should fall on the Router to implement. History should be stateless and dumb.

Router is significantly decoupled from the rest of Backbone (originally Backbone didn't even include a router) that you could probably try and test this out as a plugin first before we even have to think about replacing what we already have. I'd really like to see how history would look if it doesn't have to manage your callbacks.

[jamesplease]

Couldn't you do your async model stuff yourself in the synchronous callback, the way it is now? Most often if I'm re-rendering a route I already have the data I need. Two callbacks seems like overkill. Backbone doesn't need to be handling your fetching for you.

You're right. Backbone doesn't need two callbacks, nor does it need to handle your fetching. I think by exposing the callback that router calls, rather than inlining it, is all that it would take to make it far easier for another dev to make a plugin that implements the rest of the complexity.

I've used Ember router before and found it massively overengineered. I don't think we need to go down that, ahem, route.

Oh, yeah, for sure. There's no question that Ember-style routing covers a lot of ground, and it's a given that it's far too complex for Backbone. It may be too late to change the original post, but if I rewrote it now, I would emphasize detangling the current Router so that it's easier to create a different router on top of it, rather than pointing out the style of router I want to build on top of it :P

The URL is a pretty poor way to encode app state. It's much better to serialize to a cookie, localstorage, or the server, depending on your use case. Backbone doesn't need to handle this for you.

I don't really know what you mean? A URL is a string that represents the state of your application, whether that's a good idea or not. A router interprets this string and configures your application to be in the corresponding state.

I think nested routes is another area where things get dicey pretty quickly.

I only intended to include information about nested states to fully describe an Ember-style router. I know that Backbone would never implement them :smile: Thinking back, I prob. didn't need to go into that much detail. It was pretty late when I wrote this issue :sleeping:

but in general you don't want to go more than one or two levels deep.

This is a tangent, but I think you want to go as deep as is necessary to make your application. The current URL that I'm at is

jashkenas/backbone/issues/3653,

which is four levels if you're using an Ember-style router. Once you get the hang of ember-style routing, it's actually not even that hard to set up / reason about / test!

that you could probably try and test this out as a plugin first before we even have to think about replacing what we already have. I'd really like to see how history would look if it doesn't have to manage your callbacks.

I'll work on this and give an update when I've got one!

[jamiebuilds]

The URL is a pretty poor way to encode app state. It's much better to serialize to a cookie, localstorage, or the server, depending on your use case. Backbone doesn't need to handle this for you.

I'm interested what kind of app state you mean. I'm struggling to find a single web app that doesn't encode where you are in the app inside the url. I'm looking through all of the examples on the Backbone site and every single one works that way.

If you mean the state of an individual page for the most part devs tend to encode this with query params, and anything not important enough for query params just gets discarded.

[jashkenas]

I don't have much time, so I'll be brief:

• Alternative approaches are great, and a FullFeatured™ Router plugin that replaces the Backbone.Router is a fine idea. So go for it and see how you like it.
• I think that over-engineered routers are a terrible waste of time and sweat and tears.
• Using an over-engineered router is all about organizing your app along the lines of URLs, a brief string serialization of your application state. Paradoxically, the less rich and more HTML-y your app, the better this works. The richer and more featureful your app, the worse its state fits into nice URLs and the more useless a router is as an organizing principle.
• Using a router to organize state transitions inside of your app is fundamentally ass-backwards. You already have your app state — as real JS objects — at your disposal. Serializing them into a string state and then deserializing and reinflating them is just silly. The business about worrying about re-fetching data you already have access to is part of this.
• The idea of a Backbone Router is to drop a URL "breadcrumb" whenever your application reaches a state that the user might want to bookmark or go "back" (button) to. Nothing more than that. I think it's the right model for a minimalist library.

[akre54]

I'm interested what kind of app state you mean. I'm struggling to find a single web app that doesn't encode where you are in the app inside the url.

Location, sure. State? Nope. A URL is an entry point into your application, guaranteed to point to a resource. It isn't particularly good at storing state.

Widgets and things that don't have their own URLs shouldn't be storing their state in a router. As @jashkenas said you already have a real JS environment to work with.

The current URL that I'm at is jashkenas/backbone/issues/3653

jashkenas/backbone is the repository, issues is the resource name, and 3653 is the identifier of the issues resource. That route looks more like ':repo/issues/:issue_id'. repo just also happens to have a slash in it.

I'll work on this and give an update when I've got one!

Awesome. Looking forward!

[jamiebuilds]

You already have your app state — as real JS objects — at your disposal. Serializing them into a string state and then deserializing and reinflating them is just silly. The business about worrying about re-fetching data you already have access to is part of this.

Why would you have to discard existing objects in order to serialize the state into a url? Route-based app organization is more about code flow than anything else.

pseudo-code example:

navigate(`books/${book.id}/pages/${page.id}`, { model: page });

function pageHandler(params, options = {}) {
  let model = options.model || new Page();
  let view = new PageView({ model });
  let fetching;

  if (model.isNew()) {
    fetching = model.set('id', params.pageId).fetch();
  }

  return Promise.resolve(fetching).then(() => view.render());
}

[jamiebuilds]

Location, sure. State? Nope. A URL is an entry point into your application, guaranteed to point to a resource. It isn't particularly good at storing state.

It sounds like you are talking about individual view state rather than application state. I work on an app that is entirely "widget/card" based and this approach works really well.

Outside of global components, the state of just about every "widget" gets tossed when you route to another page. If I needed them to persist then yes I would likely use localStorage to accomplish that, but I don't see how these two things are incompatible?

[jamesplease]

This conversation about router philosophy is really interesting, and I'd love to continue it, but maybe it belongs elsewhere. I'd rather this issue focus on concrete changes to the Router that we all agree should go in v2 of Backbone.

What we do agree on (I think):

1. Backbone's Router should be pretty lightweight
2. Backbone's Router and History should be extensible
3. Backbone's Router and History are currently not implemented in a way that makes them easily extensible

The three things I currently plan to investigate changing are:

1. Expose the callback that is executed whenever a route is matched, rather than defining it 'inline' or 'in scope'
2. Expose route sorting algorithm as a separate method
3. Modify History to solely be a read-write interface to the browser url. Move other features to the Router itself

[akre54]

Sidenote @jmeas when you link to code be sure to link to a tag (or revision). The line numbers on master shift over time and make the links harder to follow in the future.

[jamesplease]

Sidenote @jmeas when you link to code be sure to link to a tag (or revision). The line numbers on master shift over time and make the links harder to follow in the future.

Good call – I always forget to do this. I'll update the links.

Updated!

[jashkenas]

The three things I currently plan to investigate changing are:

1. Expose the callback that is executed whenever a route is matched, rather than defining it 'inline' or 'in scope'
2. Expose route sorting algorithm as a separate method
3. Modify History to solely be a read-write interface to the browser url. Move other features to the Router itself
4. You can define it either inline or as a regular vanilla instance method on your router (preferable). I don't know what further "exposing" it would mean?
5. Changing the router ordering algorithm seems like the opposite of a minimal router. The ordering should be one, predictable, order. It's easy to change the order in which you define your routes.
6. Changing the balance of code between History and Router is totally a style question. There could be a nice refactor there. The historical reason for the split is because History must logically be a singleton object, whereas there's no reason that Routers need to be. That's why they're not just the same object.

[jamesplease]

You can define it either inline or as a regular vanilla instance method on your router (preferable). I don't know what further "exposing" it would mean?

Ah, sorry, not that callback – this one. I tried to include a link whenever I used 'inline' in the posts above, but I may have used the term without the accompanying link somewhere :)

You might be like, 'what that's dumb you don't need to do that,' but it just really follows naturally from number 3 on this list. The refactor makes it so there's no reason to use in-scope variables in the callback, so it can just be a separate method on the router itself. This makes it easier to test in isolation, and, for crazy people like me, it gives me a hook to override.

To be more explicit, all I'm talking about is the difference between

this.on('change', function() {});
// and
this.on('change', this.onChange);

You can always ask that I 'inline' it, which I will do, but I ask that you wait and check it out in the refactor first!

Changing the router ordering algorithm seems like the opposite of a minimal router. The ordering should be one, predictable, order. It's easy to change the order in which you define your routes.

I know that you feel this way, but some people disagree. Whether you disagree, the sort algorithm is currently tied in with the definition of what a 'handler' is (just a callback). This makes it tough to do anything with the router without overriding more code than you should have to.

Many, many developers would appreciate this method being exposed separately. If you don't want to do that, then, well, I can't stop ya. My mind wanders to Angular and Ember when I think of libraries that make it incredibly difficult to change their features, though. One of the main reasons that I (and most devs I know) use Backbone is how customizable it is!

So, once again I ask that you wait for the refactor before forming too strong an opinion to not expose this.

Changing the balance of code between History and Router is totally a style question.

Hm, interesting. I think this is analogous to saying that changing the balance between a given View and a given Model is a style question. Sure, you may get the same result, but one seems philosophically more sound to me. I think it's much more elegant if History is just an interface to the browser history API, and doesn't concern itself with what any of the routers are up to.

@akre54 said it most succinctly, I think.

History should be stateless and dumb.

@jashkenas, I'll have a preview of what I'm going for soon, which should make some of this clearer. I hope it's possible to evaluate the refactor without thinking of it turning Backbone's router into an ember router, because I do think the cleanup that I'm doing is valuable even if one intends to use Backbone's router the same exact way that it is today.

[jashkenas]

Sure thing -- more than happy to wait and see.

[nmschulte-aviture]

It sounds like you are talking about individual view state rather than application state. I work on an app that is entirely "widget/card" based and this approach works really well.

@thejameskyle, what is the distinction between these two "types" of state? What distinguishes application state from view state? In my mind (and IMO, Backbone's), there is none. Sure, supporting application state all the way down to e.g. the selection of a listing view is possibly not worth the effort, but the problem there is fundamentally the same. Why shouldn't it all be handled the same way?

Great discussion folks! I was thinking about this yesterday as well, and one point that kept popping up in my head: I should be able to use my application without even knowing there is an address bar or forward/backward navigation. Adding those bits in should be a straight-forward, "encapsulable" task. In that sense, I liken the modules we're talking about building to the principals behind Ampersand.js's components.

[jamiebuilds]

what is the distinction between these two "types" of state? What distinguishes application state from view state? In my mind (and IMO, Backbone's), there is none.

Imagine GitHub was exactly the same only it was a single-page application. The "application state" would be that you on a particular issue page, the "view state" would be that you have something typed into the new comment textarea. Both of these are persisted on page reload, but GitHub stores that information in different places.

I would challenge you to find an application that does not make the distinction of "this is application state that belongs in the url vs. this is view state that can either be discarded on page reload or stored in something like localStorage".

[gonzalovilaseca]

Any updates on this?

[jamesplease]

I don't have plans to finish it. If others want to pick up where I've left off, go for it! Otherwise, this can be closed.

[gonzalovilaseca]

I'd love too but I'm too new to Backbone, I might give it a try though. Where did u leave it? #3660 ?

[jamesplease]

Yup.