Javadoc's "first sentence" summaries are easily broken by Asciidoctor's default HTML output

[benevans]

Noticed this while working on #20 (overview file support). Thought it was my fault at first, but it's a general problem between Asciidoctor's HTML output and javadoc's generation of package overview pages.

If your package (or overview) comment (in Asciidoc) starts with anything other than a level 0 heading, then javadoc gets confused when it tries to extract the first sentence, and ends up putting <div>s in the wrong place in the final package-summary.html or overview-summary.html pages.

For example, asciidoclet's own org/asciidoctor/package-info.java currently starts with:

/**
 * = Asciidoclet
 *
 * https://github.com/asciidoctor/asciidoclet[Asciidoclet] is a Javadoc Doclet...

and gets generated correctly. The structure of org/asciidoctor/package-summary.html looks something like this:

<!-- ========= END OF TOP NAVBAR ========= -->
<div class="header">...</div> 
<div class="contentContainer">...</div>
<!-- ======= START OF BOTTOM NAVBAR ====== -->
<div class="bottomNav">...</div>
<div class="subNav">...</div>
<!-- ======== END OF BOTTOM NAVBAR ======= -->

But if I change the first line of the package comment to be plain text:

/**
 * Asciidoclet
 *
 * https://github.com/asciidoctor/asciidoclet[Asciidoclet] is a Javadoc Doclet...

...then org/asciidoctor/package-summary.html comes out like this:

<!-- ========= END OF TOP NAVBAR ========= -->
<div class="header">
    <h1 title="Package" class="title">Package&nbsp;org.asciidoctor</h1>
    <div class="docSummary">...</div>
    <div class="contentContainer">...</div>
    <!-- ======= START OF BOTTOM NAVBAR ====== -->
    <div class="bottomNav">...</div>
    <div class="subNav">...</div>
    <!-- ======== END OF BOTTOM NAVBAR ======= -->

I.e. all of the remaining content ends up inside the <div class="header">.

Something to do with the way javadoc extracts the first sentence of a comment is breaking on Asciidoctor's HTML output, causing the header div to not be closed properly. The output is still readable but the formatting is a bit off and some styling won't work since the document structure is incorrect.

It seems to be OK when the first line is a level 0 heading because this comes out in its own <h1> element, not nested inside <div>s, so I guess javadoc is able to handle it. Having said that, even these will break if javadoc's "breakiterator" option is enabled.

You can work around this by putting the first sentence of a package or overview comment in a passthrough block:

/**
 * ++++
 * Asciidoclet.
 * ++++
 *
 * https://github.com/asciidoctor/asciidoclet[Asciidoclet] is a Javadoc Doclet...

However this does mean any formatting or links you might want in the first sentence would have to be done as HTML, which is a pity!

Edit: end the sentence in the passthough block with a full stop, or surround with <p> or header tags, so that javadoc can recognize the end of the sentence correctly.

[benevans]

Out of interest I looked at what https://github.com/Abnaxos/pegdown-doclet does. It doesn't seem to be susceptible as the HTML pegdown generates is a lot simpler.

For example this markdown overview text:

First line. 

Second line.

...comes out as:

<p>First line.</p><p>Second line.</p>

...which javadoc handles fine. But given the same text, asciidoctor emits:

<div class="paragraph">
<p>First line.</p>
</div>
<div class="paragraph">
<p>Second line.</p>
</div>

...which seems to confuse javadoc when it tries to extract the first sentence.

[benevans]

Changed the issue title as this is a general issue applying to all doc comments where javadoc generates a summary from the first sentence. So overview, package, class and method summaries are affected as well.

For classes and methods, the document structure appears OK, but the summary text will be missing from the table of classes in a package overview, or the methods in a classes' method summary table.

I think javadoc just isn't expecting to see divs when it looks for the first sentence. Maybe the doclet could run Asciidoctor with simpler HTML output templates?

[benevans]

Changing a couple of output templates seemed to do the trick, should be resolved in PR #23.

[johncarl81]

@benevans, we have an opportunity to write up an article on http://asciidoctor.org/ for the 1.5.0 release, would you like to write it up? If so, drop me an email, you'll find my address on my github profile.