Eclipse Jetty Website and Documentation

jmcc0nn3ll commented 2 years ago

We are once more confronted with a problem related to the Eclipse Jetty Website.

The root of the problem stems from this issue on the Eclipse helpdesk. Nested in this issue are several objectives related to security and other factors, but the primary one I care about for this issue is that PHP support will be removed. This was after I finally broke down and migrated the site to PHP because it seemed like the most stable option after trying a handful of other options over the last several years.

We also have an issue that is opened directly for Jetty moving to the new setup here. I think this is largely on hold given feedback on the original issue, but only from timing and ultimate hosted location perspective. We might just end up hosting it through Github Pages or something...but any way we slice and dice it...

We will have to generate a static website again, and we have a handful of approaches we can take. Hugo seems to be an Eclipse-favored approach, but I am a fan of Antora since I like the folks at asciidoc, and it is another of their brainchildren.

Objectives

I have wanted to see an Eclipse Jetty website that worked as smoothly as the one we used for years that used XSLT and DocBook for combining the website and the documentation, but it has been a hard nut to crack. The primary issue has been merging the Eclipse theming with the output from our asciidoc-based documentation into something that wasn't fundamentally broken.

We need to leverage something that will allow us to generate a Website, Documentation across 4 different active branches of Jetty and host Javadoc for historical releases. It must be generated to have a nice cross-linked site between the guides, manuals, and project content. We want to minimize manually editing links and content.

Proposal

I have been working on a mini proof of concept using Antora, and I think it will work well for us. A number of other projects are using it, and it is pretty nice what can be done with it. My favorite example is the Apache Isis project. They have a custom front page, and as you go deeper, you see the telltale signs of updates to the Antora default UI.

There are some changes that we would need to make to the documentation as it currently exists in the branches today. Primarily we would no longer need to build the documentation as a part of our build. The documentation and website would be built as a separate process running from a separate repository. It uses git to clone different repositories and pull in the different branches of content to generate the site, including offering versions of the same content automatically. We could have a Programming Guide link available that would then allow the user to select the specific version they want to view, 9, 10, 11, 12, etc.

Writing the documentation would be largely the same; the same asciidoc plugins work in IntelliJ. There is just a separate convention over configuration approach to organizing content, and we would be able to keep the content history since the biggest change would be to move the jetty-documentation/src/main/asciidoc/operations-guide to jetty-documentation/modules/operations-guide/pages. The asciidoc content under that manual would remain the same structure-wise and images and things would follow a different convention.

Downsides

We would need to fork the Default Antora UI in order to theme it appropriately. There are many examples , so it shouldn't be terrible.

The biggest issue it this would all be a lot of work that I had figured we were done with a while back when I decided just to relent and use the Eclipse PHP setup.

Discussion

At this point, we have some time, and I am open to thoughts on this whole problem. If you read the original issue I linked, you will see quite a bit of backlash, but mostly against the hosted locations. I think one project came about against the loss of PHP.

jmcc0nn3ll commented 2 years ago

Currently, I have a fork of the jetty.project repository that I have started massaging into the right structure across the jetty-9.4.x, jetty-10.0.x, jetty-11.0.x and jetty-12.0.x branches. I have a repository with a copy of the antora-default-ui that I have fiddled with a bit to see how it works. Finally, I have a local directory that I can use to build the site with Antora. It requires a mess of node.js, but instructions are decent for getting that setup.

Since we don't need the contribution guide produced for every version, and the eclipse website only needs one home, I have both nested under appropriate locations in jetty-12.0.x. I think the antora-playbook (which is what generates the whole thing) can be located in a branch of the static HTML repository since that is where the final HTML will live. We could also look at using a playbook in each repository that would just generate the documents themselves, but not sure if that is worth it, writing the documentation will be a lot easier with a supporting plugin in Intellij.

joakime commented 2 years ago

+1 (from me) for having the source of our documentation be based on asciidoc (or markdown).

jmcc0nn3ll commented 2 years ago

Ok, there is a some good and bad related to using Antora. I have managed to get a few hours on this now and wanted to give an update.

I have a fork running and started bringing jetty 9, 10, 11, and 12 into alignment with the structure. It is pretty straight forward and works well in Intellij.

The structure is more rigid.

Under 9:

documentation/definitive-reference

Under 10, 11, and 12, these exist:

documentation/operations-guide
documentation/programming-guide
documentation/eclipse-website (12 only )
documentation/contribution-guide ( 12 only )

Under each of these projects, we have:

antora.yml
modules/ROOT/pages
modules/ROOT/examples
modules/ROOT/images

All pretty straight forwards for storing the content however, the examples one is an issue because Antora does not support including outside of this structure. I have worked around this in my POC by soft linking the source we include under the examples directory and committing that into git. It works.

It does not easily provide for our asciidoc jetty extension. As near as I can tell, we will need to implement that in node.js if we want that functionality.
Theming will not be particularly difficult, just time-consuming. I have used handlebars in the past, which is the templating system.
I haven't looked at the asciidoc output for a while, but the exercise has shown several errors in our current documentation. Images not referenced correctly; includes to mod files that have since moved around (in sessions mostly); some odd syntax things that I am sure must be different with preprocessing {sub-order}, etc.
The cross-references are clearer; you can xref:10.0.x@programming-guide::to get to version 10 of the programming guide, which is nice.
There is another git repo that contains a playbook in it that looks like this.

site:
  title: Eclipse Jetty
  start_page: jetty::index.adoc
content:
  sources:
  - url: https://github.com/jmcc0nn3ll/jetty.project.git
    branches: [jetty-12.0.x]
    start_path: documentation/eclipse-website
  - url: https://github.com/jmcc0nn3ll/jetty.project.git
    branches: [jetty-10.0.x, jetty-11.0.x, jetty-12.0.x]
    start_path: documentation/programming-guide
  - url: https://github.com/jmcc0nn3ll/jetty.project.git
    branches: [jetty-10.0.x, jetty-11.0.x, jetty-12.0.x]
    start_path: documentation/operations-guide
  - url: https://github.com/jmcc0nn3ll/jetty.project.git
    branches: [jetty-12.0.x]
    start_path: documentation/contribution-guide
  - url: https://github.com/jmcc0nn3ll/jetty.project.git
    branches: [ jetty-9.4.x ]
    start_path: documentation/definitive-reference
ui: 
  bundle:
    url: ../jetty-antora-ui/build/ui-bundle.zip
    snapshot: true
asciidoc:
  sourcemap: true
  attributes:
    # jetty 9 documentation attributes
    jdurl: http://www.eclipse.org/jetty/javadoc/jetty-9
    gitbrowseurl: TODO
    version: TODO
    mvncentral: http://central.maven.org/maven2
    gitdocurl: https://github.com/eclipse/jetty.project/tree/jetty-9.4.x/documentation

This addresses the concern we have had about updating documentation between releases. With this setup we wouldn't mess with releasing specific releases of documentation anymore; we just run the playbook, which fetches the branches and builds the lastest releases.

If we want versioned documentation, we could always have another playbook that pulls from release tags instead of the branches and even theme it differently if we don't want to build out the whole eclipse webtide.

My least favorite thing is that we must deal with node.js, npm, nvm, gulp, and who knows how many dependencies under all of that. Thankfully that is toolchain stuff, and the Antora community should keep their system working.

Anyway, that is my update.

jmcc0nn3ll commented 2 years ago

I should also add that there have been developments in the underlying push for all of this. The original issue has been closed and a new one opened here. There was a lot of complaining about the move and we have officially been grandfathered and our URL will not change anymore.

PHP will still be going away so we do have that to contend with.

jmcc0nn3ll commented 2 years ago

I think dropping the java side of the documentation generation would help in other ways as well. It would be nice if we could use wshito/asciidoctor-chunker to split things out by chapter that might be interesting. Assuming it can be linked into Antora as well, not sure about that.

sbordet commented 1 year ago

@jmcc0nn3ll not sure what you mean by "dropping the java side of the documentation generation", but that is generating documentation using the distribution, and running it from Ruby would need glue code. That generation is necessary for the documentation, it's just that because we use the Asciidoctor Maven Plugin, doing it in Java came natural.

jmcc0nn3ll commented 1 year ago

That would be via the maven plugin. Javadoc has to remain all the same. I am conflicted on all of this at the moment, which is why I took a break from it. It is all very neat to use Antora and then knit it all together, but I am fatigued from being forced to change things over and over. The original impetus has been pulled back on a bit so we have more time then originally thought.

I'll revisit this in the new year, tinker with it again and update my thoughts on it.

github-actions[bot] commented 11 months ago

This issue has been automatically marked as stale because it has been a full year without activity. It will be closed if no further activity occurs. Thank you for your contributions.

jetty / jetty.project