Create a meaningful diff from two AsciiDoc documents

[ahus1]

A line-by-line diff wouldn't create a document that you could render afterwards.

It would be nice to have an (HTML) rendered output of what changed.

Links:

• https://github.com/java-diff-utils/java-diff-utils
• https://github.com/DaisyDiff/DaisyDiff

[mojavelinux]

This would be a holy grail.

[ehmkah]

@ahus1 - Just for my understanding. You expect two asciidoctor files as input and the diff is also an asciidoctor file, which is displayed rendered?

Hopefully some intellij-specific-diff-view-code could be used from a plugin I wrote https://github.com/ehmkah/imgdiff which compares two image. If you do it in a similar way than you could see local changes agains a git-repository.

[wimdeblauwe]

I had the same idea a few weeks back. I had it written down to investigate if I have time. My idea is to investigate if it somehow would be possible to take 2 GIT tags, check out the diffs and generate a PDF that shows the differences. (Our main output is PDF).

The idea being that we use Asciidoc for our REST API docs (using Spring REST docs) and I would like to generate a doc that shows what changed between the last version and the current one, without having to keep track of that manually.

[ahus1]

I've given it some thought for an AsciiDoc pipeline working with either a token stream or the HTML output. TL;DR: the HTML output will produce better results in the short run and even worked with code listings.

Working on a token stream

Idea:

1. tokenize the input
2. create a diff on token level - using for example io.github.java-diff-utils:java-diff-utils

Example - I pushed it on branch try_diff_token_level here, see AsciiDocDiffTest.java: Input 1: Hello world! This **a** test! Input 2: Hello to the world! This a test! Output: Hello [.added]##to the ##world! This [.removed]##**a**##[.added]##a## test!

Giving it some coloring with CSS

<style>
.removed{background-color: #fdb8c0}
.added{background-color: #acf2bd}
</style>

results in

To improve this, some refinement would be necessary:

• attributes and includes: This will work best if all attributes would be replaced first, and all includes and other pre-processor commands like ifdef would be applied.
• tokenizing: the current tokenizer should break down headings to word level (currently they are one token) - this would improve the result.
• the highlighting works only for text, it doesn't handle changes in non-printable tokens differently (for example: if you add an xref).
• AST: results might be better (or just different) if this would run on an AST.

As the result is consumable AsciiDoc, it could be transformed into a PDF.

Working on HTML output

There is for example a JavaScript package called htmldiff-js.

For the HTML output all attributes and includes are already evaluated; this is a good thing. On the other side if the enumeration of sections changes, you'll see a lot of changes in the document.

With a minimal set of JavaScript (see https://github.com/ahus1/asciidoc-diff/blob/master/diff.js) this produces the output shown as screenshot below.

See the full file in the repo https://github.com/ahus1/asciidoc-diff.

The result could be refined by "cleaning" the diff from unwanted highlighted changes. Something this diff will not show you: when you split a paragraph into two paragraphs there will be no visible green/red highlight (the HTML element created by the library will have a zero width instead).

Let me know what you think. I'm eager to hear how this could be improved.

[wimdeblauwe]

Nice work! As I don't use HTML output, but PDF, I like the first approach better :) But lots of people are using the HTML output so that should certainly be considered as well, especially if the effort to have something useful is a lot less.

[ahus1]

@wimdeblauwe - to support the first approach it would be nice to have an Asciidoctor backend that outputs AsciiDoc. This would ensure all attributes and includes would be evaluated.

Are you aware of such a backend?

[wimdeblauwe]

Are you aware of such a backend?

I am not and a quick Google search did not reveal there is such a thing. It should be possible to write a custom backend I guess, but I don't know anything about Asciidoctor internals and I don't know the Ruby programming language. Pity it is not written in Java :)