Antora Integration

[sashokbg]

Hello I am currently working on a proof of concept with antora integration.

https://docs.antora.org/antora/2.3/install/install-antora/

Does anyone have experience with this ?

Here is a quick list of what came up so far:

• nav.adoc have to be generated manually
• default antora ui-bundle does not (yet) handle properly icons in font awesome
• it is hard to organize feature files into sub/folders in antora

Would you be interested in a documentation PR regarding this ?

[rmpestano]

Hi @sashokbg,

I would be really interested in such integration! I don't have experience with Antora but I can try to help, let me know!

[sashokbg]

Hello @rmpestano

Here is a very crude guide on how I did it (still lots of cavities):

1) Set up your java project with cucumer etc 2) Set up an antora website following this guide https://docs.antora.org/antora/2.3/install-and-run-quickstart/ 3) Create a new module in your antora project - I called it "living" for living documentation 4) Create one cucumber runner per antora page (lets say per business domain) - this is required to generate multiple cucumber.json files 5) When generating your .adoc files from the .features indicate the output directory to be the "living" directory from step 3) and do this for each feature.

        List<Feature> features = FeatureParser.parse(pathToCucumberJsonFiles);
                .....
                ..... 
        for(Feature feature: features) {
            attrs.docTitle(feature.getName());

            CukedoctorConverter converter = Cukedoctor.instance(Collections.singletonList(feature), attrs);
            converter.setFilename("docs/modules/living/pages/" + feature.getName() + ".adoc");

            converter.saveDocumentation();
        }

6) Create a "nav.adoc" for the generated pages inside "living" module <--- this steps can be easily automated with some bash

* Living documentaion
** xref:living:Calculator.adoc[]
** xref:living:Logout.adoc[]

7) OPTIONAL: Download modified ui-bundle for antora with proper awesome-font support to make icons work properly https://gitlab.com/antora/antora-ui-default/-/jobs/1023155878/artifacts/raw/build/ui-bundle.zip

8) Run antora website generation - antora --fetch antora-playbook.yaml

Here is the result:

I am available if you are interested in better documenting, or making some changes to make the integration easier.

(edit) Sample repository https://github.com/sashokbg/cucumber-asciidoc

[rmpestano]

Thank you for the detailed explanation!

Maybe we can create a SPI impl for Antora integration as we did for section layout, WDYT @andrewesweet ?

[andrewesweet]

There's a parallel with the section layout - picking which content goes on which Antora page is similar to marking sections. A thought-through SPI implementation could be used by both the default and section layouts, rendering each feature, subsection or section in a separate adoc file with nav structure following section hierarchy, for example.

I think SPI would need new interfaces exposed in order to add a new output option, "html, pdf or Antora static site".

One could argue another way, and simply add an "adoc only" output option and the SPI addin used would determine if that would result in a single adoc file or one per feature/section, for example. The Antora site would be generated separately by users in their pipelines using the content generated by cukedoctor.

An obvious follow-up feature request would be, "I don't use Antora, I use insert hot new static site generator here". Perhaps we need to consider what other integrations would require to inform where we land in the spectrum of "cukedoctor generates content only" to "cukedoctor will automate the creation of the whole site for you".

[sashokbg]

@andrewesweet @rmpestano I have started working on a first implementation of multi-page rendering based on on SPI logic as described by @andrewesweet and I would like to hear your thoughts on it.

1. I would add a new SPI called DocumentationRenderer that receives features and renders them.
2. This SPI's default implementation will be what we now find in the CukedoctorMain.java

public String render( ..required.. params .. ) {
        CukedoctorConverter converter = Cukedoctor.instance(features, documentAttributes, cukedoctorConfig);
        String doc = converter.renderDocumentation();
        File adocFile = FileUtil.saveFile(outputName, doc);
        asciidoctor.requireLibrary("asciidoctor-diagram");
        if (documentAttributes.getBackend().equalsIgnoreCase("pdf")) {
            asciidoctor.unregisterAllExtensions();
        }
        OptionsBuilder ob = OptionsBuilder.options()
                .safe(SafeMode.UNSAFE)
                .backend(documentAttributes.getBackend())
                .attributes(documentAttributes.toMap());
        System.out.println("Document attributes\n" + documentAttributes.toMap());
        asciidoctor.convertFile(adocFile, ob);
        asciidoctor.shutdown();
}

3. From here I will be able to create a new "Layout" that provides a new implementation of the DocumentationRenderer which can separate the features into different files following some logic <-- This will be in a separate pull request

Please let me know what you think. Best Regards and stay healthy !

[andrewesweet]

Hi @sashokbg. I don't have a slam-dunk winner design in my mind. The more I think about it, the trickier this is to get right. My TL;DR:

1. The current code mixes together the concerns of (1) how to turn a parse node (Feature, Scenario etc) into AsciiDoc and (2) how the resulting AsciiDoc should be converted into a final artifact (HTML, PDF, now Antora). This was never a concern in the past since one would only ever to a single call to Asciidoctor.convertFile. In your new world you care about this distinction. You'll probably have to refactor to account for this.
2. Whatever the solution is, it likely involves the renderers returning more structure data than just a String containing the rendered AsciiDoc so that higher level entities can make decisions about how to convert that AsciiDoc into a final artifact. Or...
3. Going the other way and pushing more or all of the responsibility for converting to the final artifact into the renderers.

I shall think on this further.

My stream-of-consciousness is below.

I'm not convinced CukedoctorMain (refactored into a DocumentationRenderer or otherwise) is the right place to make the decision as to which content goes into which file. It gets the AsciiDoc representation of the input features from the call CukedoctorConverter.renderDocumentation(), which returns a String. How will one split that String out into multiple Files, each representating a Feature, in a reliable way? How will one decide what to name each? How would one produce a navigation/content/index page?. One would have to parse the AsciiDoc string. This seems unnecessarily abstruse given we already parsed the Cucumber output into a tree-like structure with Iterable at the root.

We also must ask how much flexibility we want to grant plugin authors. Should I be restricted to rendering into files at the Feature level? Why not Scenarios or Steps? To keep it simple, I'd start with the statement that we mirror Gherkin itself and at most all a file-per-Feature, but not per-Scenario. This could well be restrictive for future use cases, but I don't feel I understand such use cases well enough today to produce a useful design for them.

However one cuts it, I think CukedoctorConverter.renderDocumentation() (or its future replacement) will need to return some more structured data than a String. The most naive example would be Iterable rather than today's String. This works well enough for rendering to HTML, say. I could choose This is almost certainly insufficient, however. For example, it does not solve the naming problem in a way that is useful for writing a content/index/navigation page.

This itself raises the question of just how separate are the concerns of (1) rendering a Feature/Scenario/whatever to AsciiDoc and (2) converting the resulting AsciiDoc into the final artifact (HTML, PDF, Antora website). I'd like to think these are two separate concerns, but I don't think this is true. There are conditionals around "do special things if I'm writing to PDF" in CukedoctorMain, CukedoctorConverterImpl and CukedoctorFeatureRenderer, for example. As another example, today we start building AsciiDoc with heading level 0 in CukedoctorConverterImpl, bumping the level as we go into each section ("Summary","Features") and then each feature. However, for Antora, you'd probably want each Feature title rendered as heading level 0 in its own page, meaning you'd need your own version of CukedoctorConverterImpl where the call to FeatureRenderer.renderFeatures() passes a new CukedoctorDocumentBuilder rather than creates a nested one, or you have a FeatureRenderer implementation that ignores the provided builder and creates its own.

[rmpestano]

Thank you guys for the detailed and great ideias!

I think we may need a new module like antora-converter which would have its own renderer a d converter. We can reuse/extract some useful parts from cukedoctor-converter like feature search and parsing to a cukedoctor-common module.

Just an idea because depending on the Antora impl and what output it needs to generate we may have to change/refactory/adapt so many things in cukedoctor-converter that Its easier to create a new converter specific to Antora

We may also need another main/cli app for Antora because cukedoctor-main is quite tied to current converter, e.g the parameters: hideSummarySection, hideStepTime, hideTags, hideScenarioKeyword etc..

[andrewesweet]

You are probably right, @rmpestano. The way one must render features to Asciidoc and how one uses that Ascidoc to produce a consumable artefact are coupled.

One way to increase reuse might be to break apart FeatureRenderer. It was two responsibilities right now; render an individual feature and render a list of features. The former is potentially reusable across many converter implementations as a plug-in. The latter is probably converter-specific e.g. produce one file per Feature for Antorra vs one big file for existing implement. The latter therefore arguably a responsibility of CukedoctorConverter, or a new intermediate renderer - FeaturesRenderer (with a better name, I hope).

[sashokbg]

I agree that it looks hard to split into multiple pages the way it is currently working. Do you think another solution would be to "wrap" the current project into another that handles multiple pages ?

Something similar to what I wrote in my original post. A piece of code that just calls multiple times the current implementation and then assembles it to pages.

[andrewesweet]

That seems plausible. A CukedoctorConverter implementation that selectively passes say single _Feature_s to the existing implementation or FeaturesRenderer implementation (you'd get the AsciiDoc "headers" and existing Summary section once for each Feature in a naive implementation, for example). This implementation would then write any cross-feature pages (like a navigation page or some such).

[rmpestano]

The more I look into cukedoctor-converter module the more I think we need a new module for Antora, both for parsing and generating the adoc files. For example, cukedoctor-converter uses these asciidcoc extensions to modify generated HTML. For example, to have a minimizable feature section we insert minmax macro into generated adoc, see here. Of course, we can disable these extensions but I think a fresh dedicated module for Antora will be much cleaner easier to maintain solution.

I propose that we start cukedoctor-antora module and extract common useful parts from cukedoctor-converter module to a new module called cukedocto-common.

[rmpestano]

Actually, we can have a new renderer instead of disabling the min-max extension but still, when I look into generate documentation with summary renderer, intro, pdf custom theme, I think it can be better to have a separated module.

Maybe we can start as a new renderer and then later extract to a separated module if it's too hard to use inside cukector-converter, WDYT?

[rmpestano]

A piece of code that just calls multiple times the current implementation and then assembles it to pages

Yea it can work. Also we can have a new impl for CukedoctorConverter if needed, we would have a new factory method for Antora here

[sashokbg]

@rmpestano @andrewesweet I have just opened a very crude draft PR https://github.com/rmpestano/cukedoctor/pull/179 that adds a completely new converter (MultipageConverter) which uses components from the main cukedoctor-converter project and which provides 2 SPIs for now:

• MultipagePager to dermine how features are grouped into pages
• MultipagePersister to determine how and where the documentation is written

Please let me know what you think of the draft and if it is going in the appropriate direction. If it looks OK to you I will gladly spend some time to add tests and clean up the code.

[rmpestano]

Hwy @sashokbg. it looks great to me. I didn't run it locally but I think we should disable summary, steps time and all the extensions like (styles, footer, search etc...), are you doing that?

I'll have a deeper look and try it over the weekend, thank you very much for the idea and contribution so far!

[sashokbg]

@rmpestano Hello and sorry for the late answer - I was busy at work the last 2 works. I have added some unit tests and BDD tests to the PR as well as the possibility to pass document attributes like we do with the classic converter.

[rmpestano]

Thank you for all the work on this @sashokbg!

I've just released v3.7.0 with the multipage extension, let's keep this issue opened until we have the full Antora flow described by you here implemented.