Support Next/Previous links

wbamberg commented 4 years ago

In MDN writers want to link together pages that are presented as a linear sequence of chapters, typically by providing "<< Previous" and "Next >>" (and sometimes "^ Overview") links.

This basic feature is reimplemented independently by three groups of KS macros:

Next, PreviousNext, Previous: used by various sets of pages including the JS guide and Games docs
NextMenu, PreviousMenuNext, PreviousMenu: used by the Learn docs
nextPage, previousPage: used by no en-US pages, and a hundred or so outdated pages in other locales

The table below summarises how many pages use these macros:

Macro              en-US           all locales
----------------------------------------------
Previous            11                 61
nextPage             0                 91
previousPage         0                 96
PreviousMenu        20                133
Next                14                163
NextMenu            23                221
PreviousMenuNext   129               1025
PreviousNext       105               1066

The bulk of them are translations of the Learn area, which seems like important content.

Assuming this is a feature we want to support in stumptown, the obvious suggestion is to use the "chapter_list" feature which we already have. This feature enables writers to say that a collection of pages live together and should be presented in a particular order. They do this by providing a special YAML file listing the pages, like this: https://github.com/mdn/stumptown-content/blob/master/content/en-US/html/guides/Guides.yaml.

Currently these things are used in landing pages and sidebars. We could use them for Next/Previous, like this:

build-json notices that a page is listed in a chapter list. It gets the previous and next item in the list and adds next_page and previous_page to the JSON for the page.
stumptown-renderer sees that the page JSON has next_page and previous_page items, and adds the appropriate navigation elements

This seems nicely coherent with the other uses for chapter lists, and means authors get next/previous for free, without having to do anything else. And it makes a nice split between content and rendering: writers don't have to decide how next and previous should be rendered, they just say what they should be.

This does imply that a page can only belong to one chapter list, which seems reasonable.

Acceptance criteria

The build-json code adds "Next" and "Previous" links to pages that are included in a chapter list.
If the chapter list also includes an "Overview" link, then the build-json code adds that, too.

wbamberg commented 4 years ago

Finding chapter lists

This proposal means build-json needs a way to find chapter_list files. At the moment it doesn't need this because they are always pointed to by something else (sidebar and landing page specifications). But in this proposal they are not. It seems like the implementation needs to go something like:

find all the chapter lists
remember the pages they contain
when processing a page, if it's in your set from (2), find and add Next/Previous (and Overview or Up, if we want to enable that too)

At the moment chapter lists are just scattered through the tree in what seem like relevant locations (for example, the chapter list for HTML guides is in the parent folder for the HTML guide pages). Although they do have a special format, they don't contain any metadata saying what they are.

So I can think of a few approaches:

pages that are in chapter lists point to the chapter list they are in. I don't like this because it's redundant and offers opportunities for errors: what if the chapter list is updated to remove the page, but we forget to update the page?
we add some metadata to chapter lists to say what they are: then build-json can scan the tree looking for these things. I don't think it's right to call "chapter_list" a recipe type (recipes describe pages in stumptown). And I don't like the idea of adding a new metadata item, or a new concept of "the sort of thing that this file is" that's distinct from recipes. I think it would be confusing.
we corral chapter lists in their own bit of the tree. This is basically what we have done for "related_content", which seems like a quite closely-related concept. The main drawback is that chapter lists are no longer kept close to the pages they contain, which might make them harder to maintain.
we drive build-json from related_content. Instead of just scanning /content to look for pages to build, it picks up a related_content file for a particular area (JavaScript, say) and uses it to find all the pages it needs to build, so as to build "the JavaScript docs". I think this is an interesting idea but it is a big change and I'm not sure what all the implications are.

Unless we wanted to dig into (4), then I'd favour (3).

wbamberg commented 4 years ago

Scraping chapter lists

It's going to be tricky to scrape documents that use Next/Previous into the model proposed here, because the Wiki docs don't provide enough information for the scraper to build sensible chapter lists.

Perhaps we could:

manually create chapter lists in stumptown, which might be practical if there aren't too many of them
just remove these macro calls when we scrape the pages (note: this is different from triaging them as "remove", because we would not remove them in the Wiki version)

The next step to explore this option would be to get a sense of how many chapter lists there might be. We could have some help to construct them (for example, GroupData.json includes an optional "guides" section for each API, which should just map directly to a chapter list).

chrisdavidmills commented 4 years ago

Nice writeup Will!

A few points/questions:

So you are saying that the code that builds these links would have an option to include the "^ Overview" type links, or not, depending on your needs? I think this is a good idea. Also, should the overview page that sits over the page series have a generated link put on it optionally/automatically, along the lines of "Next >>" or "Start series >>"?
I think we could stand to not put these on all translations, and just let them lose this functionality. We are planning to ditch most translations anyway, and losing these links wouldn't make the pages unusable, just slightly less convenient to use.
I think option 3 sounds reasonable, as long as there is some link or lookback command you can use to easily find what chapter_list generates which set of page's links.
Scraping the chapter lists - in some cases at least (such as learn), couldn't you just generate the chapter lists from the structures contained inside the learnsidebar macro? Or from the lists of articles in the body of the module landing pages?

ddbeck commented 4 years ago

we add some metadata to chapter lists to say what they are: then build-json can scan the tree looking for these things. I don't think it's right to call "chapter_list" a recipe type (recipes describe pages in stumptown). And I don't like the idea of adding a new metadata item, or a new concept of "the sort of thing that this file is" that's distinct from recipes. I think it would be confusing.

Seems like a similar problem to examples. We need a way to distinguish between types of content that aren't pages (different kinds of examples, chapter lists, who knows what else in the future). I don't have a solution to this problem, but I'm not sure dismissing option 2 makes this confusing thing actual go away.

wbamberg commented 4 years ago

@chrisdavidmills

So you are saying that the code that builds these links would have an option to include the "^ Overview" type links, or not, depending on your needs? I think this is a good idea.

I didn't really explain how this might work for "Overview" pages. In the current implementation, chapter lists don't actually have the concept of an overview page, but this "Overview/Next/Previous" use case makes me think that they should. I'm not sure if overview pages should be an optional element in a chapter list or a mandatory element.

So, I think that if a page is in a chapter list, and that chapter list includes an overview page, then build-json will add an "overview_page" link into the JSON that it builds for the page. Then the renderer can render this as a link like "^Overview" or whatever.

Also, should the overview page that sits over the page series have a generated link put on it optionally/automatically, along the lines of "Next >>" or "Start series >>"?

That makes sense.

Scraping the chapter lists - in some cases at least (such as learn), couldn't you just generate the chapter lists from the structures contained inside the learnsidebar macro? Or from the lists of articles in the body of the module landing pages?

Yes, but this would be very specific to the particular chapter list, and wouldn't generalize to other areas such as the JS guide or the Games stuff, or...

@ddbeck

we add some metadata to chapter lists to say what they are: then build-json can scan the tree looking for these things. I don't think it's right to call "chapter_list" a recipe type (recipes describe pages in stumptown). And I don't like the idea of adding a new metadata item, or a new concept of "the sort of thing that this file is" that's distinct from recipes. I think it would be confusing.

Seems like a similar problem to examples. We need a way to distinguish between types of content that aren't pages (different kinds of examples, chapter lists, who knows what else in the future). I don't have a solution to this problem, but I'm not sure dismissing option 2 makes this confusing thing actual go away.

To me it's a little different from examples. The problem with examples is knowing which sort of example an example file contains (executable or static), which feels like a problem contained within the code-examples domain, so I'm more comfortable with a specific solution. The problem here is knowing that a given YAML file is a chapter list at all and not something completely different. That feels like a more generic problem, so tacking on a specific solution feels more sketchy.

chrisdavidmills commented 4 years ago

I didn't really explain how this might work for "Overview" pages. In the current implementation, chapter lists don't actually have the concept of an overview page, but this "Overview/Next/Previous" use case makes me think that they should. I'm not sure if overview pages should be an optional element in a chapter list or a mandatory element.

I think in most cases, it makes sense to have an overview page, even if it's just a collection of links to group the members of the list together. It'd be interesting to have a think about cases where we don't have overview pages, and if that's ok, or even desired?

Yes, but this would be very specific to the particular chapter list, and wouldn't generalize to other areas such as the JS guide or the Games stuff, or...

Yeah, good point.

a2sheppy commented 4 years ago

This really sounds great so far. I agree that having the ability to create an overview page, ideally with some introductory text but at least a table of contents of the pages within, would be useful.

I suspect also that there are many places we should be using pagination buttons like this where we are not.

mdn / stumptown-content