Support for Generated Navigation

[pvande]

As some background, I'm looking to make docs.page work with automatically generated API documentation. The tool that I'm using to generate my docs is aware of the hierarchy being modeled, and can express that as JSON, but generally shouldn't be responsible for writing or updating the docs.json file.

I'd ideally like to be able to manually maintain my own docs.json file — allowing for control over title, description, and some general navigation — and "embed" the generated navigation details myself.

Strawman proposal:

// docs.json
{
  "name": "Example Project",
  "sidebar": [
    {
      "pages": [
        {
          "title": "Welcome",
          "href": "/"
        },
        {
          "title": "Getting Started",
          "href": "/"
        },
        {
          "group": "API Docs",
          "load": "/docs/api-sidebar.json",
        }
      ]
    }
  ]
}

// docs/api-sidebar.json
{
  "pages": [
    {
      "title": "Foo",
      "href": "/api/foo"
    },
    {
      "title": "Bar",
      "href": "/api/bar"
    },
    {
      "title": "Baz",
      "href": "/api/baz"
    }
  ]
}

I can also see this being useful for variables and tabs as well, possibly elsewhere.

[Ehesp]

The current limitation is that docs.page works with a runtime model in mind, whereby it queries the information it needs for the specific page being loaded - the downside of this is that it's not aware of "other files" outside of that request.

This means we'd need to first grab the config + markdown, parse the config, and query the API again to grab any additional files (e.g. docs/api-sidebar.json).

This isn't impossible, but I'm somewhat conscious of overhead from API roundtrips. The duel or alternative approach I'm currently exploring is (tldr version):

1. Get the SHA of the current request
2. Check cache for that SHA
3. If we don't have anything, grab everything we need from GitHub (all files from docs + config), bundle and cache it
4. Return the response

The benefits of this are:

• We can access all docs files without hitting a slow API
• Enables the sort of things you're on about, and also https://github.com/invertase/docs.page/issues/393
• Subsequent requests will be performant, since it's just hitting cached results
• We could also check if no ref (e.g. HEAD) is being requested, and return older cached results whilst in the background it gets new content (stale while revalidate approach).

The cons vs the current approach are:

• First requests will most likely be slower
• Additional costs (storage, cache)

[pvande]

I definitely appreciate the cost of an API roundtrip here, but I'm not convinced that would exclusively be a bad thing. For example, being able to chunk very large hierarchies has the potential to improve rendering times for sidebars with thousands of pages. (Truthfully, a similar approach seems as though it could improve times on any large resource, and superficially doesn't seem dissimilar to how embedded assets are handled.)

The approach you're describing also seems like it would work, though eagerly populating the cache also comes as a drawback for large documentation sets.