improve sidebar navigation in jekyll

[hossman]

at the moment, the jekyll theme we're using enforces only 2 levels in the leftnav sidebar navigation, and "folders" in the sidebar can only contain links, they are not themselves links.

The BuildNavAndPDFBody code that currently generates the sidebar.json is fairly simplistic (and not very usable) in order to play nice with these rules:

• every immediate child of the main page (ie: "top level page") is a folder
• every top level page is the first link in it's own folder
• all descendent of top level pages are listed in sequence in the folder

(it was largely just ment to be a proof of concept of building the sidebar.json programatically from document metdata)

@ctargett mentioned at one point in the past that we could potentially:

• have multiple sidebars (potentially one for every top level page?)
• change the theme to support sidebar(s) that goes deeper then 2 levels

we should look into those options and figure out what we think makes the most sense for us, at least in terms of something useable for the rollout - obviously we could tweak layout/style until the end of time.

[ctargett]

It looks to me like there is support for another level of nav in the _layouts/sidebar.html file - I'm going to make a branch to experiment & demonstrate how to make it work.

The multiple sidebars is designed to have different pages/sidebar for different "products" - so, if we had multiple Guides we wanted to produce, we could make different sidebars for each one. This would be a more complex solution, so I'll first see how the extra level works out.

[ctargett]

One difficulty to note here is that the framework for page hierarchy is totally different between Confluence and the Jekyll-based approach.

In Confluence, a page has a parent ID to indicate its place in the hierarchy. So, you have a page with content, and it has a child page with content, etc. That allows a nav like this:

Home page
   Section 1 page (text)
      Child page (text)
      Child page (text)
         Another child page (text)
   Section 2 (text)

With Asciidoc and Jekyll, however, there is no hierarchy unless you impose one (which you have a solution for). The sidebar for the theme we're using doesn't understand any hierarchy, so we need to construct it. It by default only supports something like this:

Home page
   Section 1 Title (only for sidebar)
      Section 1 page (text)
      Child page (text)
      Child page (text)
      Another child Title (only for sidebar)
            Another child page (text)
   Section 2 Title (only for sidebar)
      Section 2 page (text)

This is a shift, that may inspire us to make some changes down the road. I'm still figuring out how this might be able to work, but wanted to make sure this observation is front & center in the sidebar discussion since it is a shift.

[hossman]

yeah ... i keep forgetting about the "section" vs "page" distinction in the jekyll sidebar stuff.

A couple of straw man ideas that would be easy to implement in the code that builds the sidebar/pdf...

• the code could look for metadata in a page telling it what the "Section title" should be when that page is the first page in a section (and it could error if you re-org pages and the first page in a section doesn't have a section title, or a page has a section tile but isn't the first page in that section)
• we could abandon the idea of building sidebar.json from the individual page metadata, and instead switch to having a "scaled down" version of sidebar.json (maybe call it "nav.json") that people edit:
  • each section would only contain:
  • the section title
  • the (ordered) list of either subsections, or file names of adoc files
  • ...and then the build system could use nav.json to:
  • create the "real" sidebar.json by pulling the in the page titles for each page
  • building the PDF by writting an adoc file with all those page files in that order -- like it already does -- but also adding the section titles (with the expected header level) in the appropriate places
  • error if any files don't exist, or if any "extra" files exist in the directory w/o being listed in nav.json

(my primary concern with the sidebar vs pdf includes has always been thta i think it owuld be a really bad idea to have to worry/remember about editing 2 diff things when you want to add/move/remove a file in the guide, or when you want to change the name of a file/page ... i thought some basic metadata in individual parent pages about what their children were, and in what order to list them, would be the simplest way to do that -- but it doesn't really account for the sidebars "section title" concept, so if there is an easier way to do it, or if folks think editing a "nav.json" makes more sense, i'm certainly open to that)

[ctargett]

In the branch "sidebar-nav" I have an example working with an additional level of nav.

The structure of the data file is like this (I used YAML only because it was a little bit easier to work with since I didn't yet know what "level" things needed to be on - I think we'd prefer JSON, though):

entries:
- title: sidebar
  product: Apache Solr Reference Guide
  version: 6.2
  folders:
  - title: Using the Solr Admin Interface
    url: using-the-solr-administration-user-interface.html
    folderitems:
    - title: Collection-Specific Tools
      url: collection-specific-tools.html
      subfolders:
        - title: Analysis Screen
          url: analysis-screen.html
    - title: Core-Specific Tools
      url: core-specific-tools.html
      subfolders:
        - title: Overview Screen
          url: overview-screen.html

In the sidebar-nav branch this is in jekyll/_includes/nested.yml.

There are a couple of caveats to this:

• When a main section is opened, all of the sub-levels open also. This allowed the 3rd level to be direct links instead of Title-then-Link, which would have been yet another level. We'll have to see how this scales.
• The navgoco plugin wants to add "display: none" into the HTML once data-index levels get beyond 3. I think I've satisfactorily got around this by inserting CSS to force display of all data-index levels.

I also simplified the sidebar.html a lot, removing the conditions for document type ("web" and "pdf"), and checks if the URL is "external" or not (which is impossible in our current use case).

I still have not solved the problem of having the top level click-able AND expandable. I'm at the limit of my non-existent knowledge of JavaScript - I think someone who knows that even a little is going to need to further improve this.

But, at this point, we have a way to add a 3rd level. I don't know if we can go deeper than 3 levels, though.

If this doesn't work, there are two other models we can look at:

• The SendGrid docs have a nice nav, and they use Jekyll. They've done this with a custom Ruby plugin. More details: https://github.com/sendgrid/docs or the live site at https://sendgrid.com/docs.
• I have access to a proprietary approach that I can borrow from that uses 2 data files - one for the main headings, and another for the nested nav. The nested topic data file is looped over. This ends up actually implementing several nav "sections". I'm not a huge fan of it, but I can dig into it and see how much we can borrow to make work for our needs.

[ctargett]

Meant to upload a screenshot of what the current state looks like:

[hossman]

I haven't had a chance to take a look at this branch yet but a few responses...

I still have not solved the problem of having the top level click-able AND expandable. I'm at the limit of my non-existent knowledge of JavaScript - I think someone who knows that even a little is going to need to further improve this.

I know this is possible, and i've even done some hand rolled javascript before such that clicking on a div once expanded it, clicking again took you to a link -- but yeah, it should also be easy to make clicking the words in the box take you to the link (and presumably auto-expand that box when the page loads? ... does that work currently on your branch?) but clicking anywhere else in the box -- including the triangle -- just toggles the expand/collapse ... i can try to muddle through that later unless/until someone with non-trivial javascript knowledge steps up instead.

But, at this point, we have a way to add a 3rd level. I don't know if we can go deeper than 3 levels, though.

IIRC we only have a handful of pages that go deeper then 3 levels (check the generated pdf-main-body.adoc and look for leveloffset > 2 (we start counting at 0) ... we could always make the tool that parses the page and builds the sidebar fail if the tree gets deeper then 3, forcing doc writer to re-think where the page should live (ie: make it a sibling instead of a child)

[hossman]

Updates to the branch...

• the ant build-nav code now generates the new style sidebar.json (and i switched the jekyll code back to using it instead of the hand made nested.yml demo
• clicking a 'folder' name in the sidebar loads that page, clicking the caret to the right of the folder name expands but stays on the current page (very similar to how confluence currently works)
• the html/javascript for the sidebar now ensure that the current page is always expanded \ this is done by comparing the page.shortname of the current page with the href of the sidebar entries -- no cookie needed (again: very similar to how confluence currently works)

Now that i understand the sidebar templating and "navgoco" jquery plugin a bit better, i'm 90% confident we can support arbitrarily deep nested sidebwars -- even if we decide that's a bad idea, and we want to cap it at a fixed number, we should still be able make the levels of the sidebar behave consistently (ie: only expand the 2nd level when that's the path you're looking at or a child page, etc...)

FWIW, here are the current pages that go deeper then 3 levels...

build-nav:
   [delete] Deleting: /home/hossman/lucid/refguide-asciidoc-poc/confluence-export/converted-asciidoc/_data/pdf-main-body.adoc
   [delete] Deleting: /home/hossman/lucid/refguide-asciidoc-poc/confluence-export/converted-asciidoc/_data/sidebar.json
     [java] Building up tree of all known pages
     [java] Creating /home/hossman/lucid/refguide-asciidoc-poc/confluence-export/converted-asciidoc/_data/pdf-main-body.adoc
     [java] Creating /home/hossman/lucid/refguide-asciidoc-poc/confluence-export/converted-asciidoc/_data/sidebar.json
     [java] WARNING: depth==4 for basic-authentication-plugin.html
     [java] WARNING: depth==4 for kerberos-authentication-plugin.html
     [java] WARNING: depth==4 for rule-based-authorization-plugin.html

[hossman]

Now that i understand the sidebar templating and "navgoco" jquery plugin a bit better, i'm 90% confident we can support arbitrarily deep nested sidebwars ...

...i was too cocky -- liquid doesn't support recursion, so this isn't actually possible.

for now i've implemented 4 levels deep since that's what we actually have in the ref guide. ant build-nav will fail if someone tries to add a page deeper then that.

(Personally i kind of think it looks cleaner w/only 3 levels, so after we migrate we might want to consider re-orging those few pages that are depth==4 and tighting up that restriction)

In general i think the current sidebar looks much better, and behaves much better as you click around. see the messages from the last 2 commits on this branch for some details on what changed, but the bottom line is I think what we have now on this branch is good enough to go live with, and tweaking the CSS later to adjust things should be fairly easy now.

@ctargett - what do you think of how thinks look at the moment, and how the sidebar a a whole behaves as you expand/collapse various sections of the accordion?

[ctargett]

That looks pretty good. I'm glad you were able to fix the staying-open-to-the-page-you're-on problem and the highlight-the-page-you're-on problem in one shot.

Probably took you longer than one shot, but it looks good and I agree it's good enough to go live with. If you're ready, we can merge the branch - I can or you can, whichever you prefer.

[hossman]

merged to master