ctargett / refguide-asciidoc-poc

Proof of concept of Solr Ref Guide converted to asciidoc format & using Asciidoctor for publishing
2 stars 4 forks source link

= Solr Ref Guide in Asciidoc :toc:

This project is a proof of concept for reimplementing the https://cwiki.apache.org/confluence/display/solr[Apache Solr Reference Guide] as an Asciidoc-driven HTML site.

== ** AS OF 10 MAY 2017 THIS PROJECT HAS BEEN MERGED WITH SOLR SEE https://git1-us-west.apache.org/repos/asf?p=lucene-solr.git FOR LATEST CODE

OLD README CONTENT FOLLOWS

Instead of the current approach that uses Confluence, HTML pages and PDFs could be generated using a publishing toolchain that is integrated with existing Lucene/Solr build processes.

Asciidoc has been chosen as the format for Ref Guide format. This is a plain-text light-weight markup language, similar to Markdown, but with more options for structured articles, manuals, books, etc.

See the wiki page in this repo https://github.com/ctargett/refguide-asciidoc-poc/wiki/Why[Why Asciidoc] for discussion of reasons to change from Confluence and why Asciidoc is a good choice.

== What's in this Repo? This repo is a proof-of-concept for converting the Ref Guide to Asciidoc format. It includes:

== Site Generation Process

Asciidoctor is designed as a toolchain for publishing content. Confluence provides a similar toolchain, so when thinking about replacing it, we must think about each aspect of the process and how it can be replaced.

Confluence provides a full end-to-end solution, from editing content to publishing a PDF. It includes features such as page comments and search, which are vital to community involvement and successful navigation of a topic as broad as Solr.

This project proposes the following:

More on these options below - for specifics, see solr-ref-guide/README.adoc.

=== Asciidoctor

Asciidoctor is a toolchain written in Ruby which facilitates converting text files to other formats for publishing.

Asciidoctor is both the markup language AND the conversion tool. The https://github.com/asciidoctor[Asciidoctor GitHub repositories] include several projects for build integration, content preview, content styling, and other tools required for a robust publishing solution.

=== HTML

It's possible to use Asciidoctor to generate HTML pages. However, the HTML output is limited to what is defined in the source content itself. There is no ability to add site navigation or add customized JavaScript. These more robust features can be added with a static site generator like Jekyll which is what we will be using.

=== Jekyll

Jekyll is a static site generator, which uses templates to create complete websites from raw content files in Asciidoc or Markdown (and other simple markup formats).

Jekyll is the most well-known of currently available static site generators, and has the most robust development and user community, and is the most mature.

Jekyll is written in Ruby and depends on native libraries, so it won't be viable to use JRuby to invoke jekyll via java, so building the HTML ref guide site will require that Jekyll be installed locally beforehand.

We're also using the https://github.com/asciidoctor/jekyll-asciidoc[`jekyll-asciidoc] plugin, which would also need to be installed locally as a dependency. Thebuild.xml` file still needs to be updated to try and detect if these gems are installed as part of the build process, and give a clean error w/install instructiones if they are not.

The current POC uses a Jekyll https://github.com/tomjohnson1492/documentation-theme-jekyll[Documentation Theme] developed by Tom Johnson, so the POC has a UX that is very different from out-of-the-box Jekyll.

I have additionally modified Tom's theme quite a bit:

The borrowed theme implements "search", but it is not a full-text search engine. It's a keyword index of titles and page description metadata.

TO DO

=== PDF

We'll use the asciidoctor-ant plugin which includes https://github.com/asciidoctor/asciidoctorj[`asciidoctorj] which itself includesasciidoctorj-pdf, an implementation of https://github.com/asciidoctor/asciidoctor-pdf[asciidoctor-pdf`] an Asciidoctor project to generate PDFs.

This means that even without Jekyll (or Ruby) installed, anyone will be able to build the Ref Guide PDF using jars that can be obtained from maven central via ivy.

There are some interesting issues that still need to be overcome:

As for styling, a YAML-based theme file is required to define colors, fonts, sizes, etc. This implements many of the features of CSS.

Overall, this plugin is the easiest to use of the various options (another option requires converting the content to DocBook format first, then to PDF), but is still in an alpha stage (as of 10 Nov 2016), so many features are still pending for future releases.

TO DO

== Comments

Comments are one of the two main reasons why a static site generator is required to have a full-featured replacement for Confluence.

This POC uses the Apache Comment System. See this repo's wiki page https://github.com/ctargett/refguide-asciidoc-poc/wiki/Comments[Comments] for more details on this system.

Because static site generators are template driven, it's simple to add JavaScript snippets to the template for each page. Variables allow filling in a page ID (this POC uses a page shortname) as each page is generated, which pull in the comments from the comment system.

Migration of existing comments from Confluence was briefly considered and rejected as too complicated. Comments are only available from the XML export from Confluence, while we needed the HTML export for effective content conversion. Then there is a question of if they are worth migrating - my own view is that they are not.

For more on how these decisions were made, see this repo's wiki page, https://github.com/ctargett/refguide-asciidoc-poc/wiki/Comments[Comments].

TO DO

== Open Questions

=== Location of Source

Should the content source live in a separate tree?

Should the content source live in a new sub-directory of the Solr Git repo?

=== Organization of Files

How should we organize the Ref Guide pages in the directory tree?

Some examples of how others have done it are available in this repo's wiki page https://github.com/ctargett/refguide-asciidoc-poc/wiki/FileOrganization[File Organization].

=== Hosting Options

Without Confluence, we will need to determine how and where to host the rendered pages. Some initial ideas:

. Host in ASF CMS with website. . Host however the javadocs are hosted.

=== Search

How will we provide search?

Recommend probably indexing generated HTML pages. Could use bin/post from Solr to recurse over the HTML files and index them. In this case, we will need to figure out where to host Solr.