Create `@example` "include" pre-processor feature for java/scala-doc

[ktoso]

We want to use this to power examples for all directives. It will make it easier to maintain the directive docs and would also address https://github.com/akka/akka/issues/18660

The idea is to keep examples somewhere where they are compiled as usual. Then we'd put a reference to an example directly into the scaladoc and then have a preprocessor (on sbt or scaladoc/javadoc level) that includes the snippet at that point into the scaladocs.

Example would be:

class ThingDirectives {

/**
 * Does things.
 *
 * @example $root/akka-http-docs/blabla/ExampleDirectivesSpec.scala#example
 */
def example(t: String): Directive0 = ???
}

It should:

• Fail compilation if include does not exist
• Work for Scaladoc and Javadoc
• create a <code> block basically
• in Scaladoc there was an example feature I believe, check that
• allow us to move all directive method docs into scaladoc, so we don't have to maintain explicit docs pages for each of them

[jonas]

References #187

I'm interested in working on this. Should I try to land the tooling upstream in Paradox or do we want to iterate on it in this project before suggesting a fix for lightbend/paradox#26

[jonas]

In my investigation so far I didn't manage to get a Scaladoc preprocessor working. These are the main examples I've looked at so far:

• https://github.com/szeiger/extradoc
• https://github.com/todesking/sbt-custom-doclet and https://github.com/todesking/scala-json-doclet

I wonder if for the Scala directives it would be in scope to use scala.meta or the approach of sbt-doctest to extract the directive scaladoc and the directive signature? That is we would be able to copy paste the markdown to the scaladoc but keep the directives documentation more or less as it currently is. This was why I created lightbend/paradox#26 although I should probably have discussed this here first.

[jrudolph]

I think we should do the processing directly on the source files (probably through sbt). It is important that the published source artifact also contains all the examples because many if not most people will read the scaladoc either through an IDE (which will pick up the documentation from the source files) or by reading the source code to which the IDE will link to.

[jonas]

First prototype for the scala side is here using jonas/akka-http@14b744a2dbc76812a4e78a698cdd7df3ca6d3b49

A couple of notes:

• Basic directives should be split into multiple groups to match its overview page
• In the Scaladoc basic directives comes first but we should probably tweak groupprio to use the order in the listing by trait
• Navigating the directive docs is somewhat painful since you completely loose context (at least in Chrome) when clicking a link to another directive and go back afterwards.

Next step will be to generate that code and feed it to unidoc.

[2m]

It would be interesting to see if the new scaladoc (example of the new output here) might fix some usability issues.

[jonas]

I will try. That would be a good excuse to configure cross building for Scala 2.12.

[jrudolph]

:+1: An old version of the akka docs generated with 2.12 can be found here: akka-docs.virtual-void.net

[jonas]

OK, got a bit further using scalameta to extract signatures and comments. This would require to have the tooling separate from sbt (until version 1.0) since scalameta only supports 2.11+.

[2m]

@jonas, scalafmt had the same issue, where it was not possible to use scalafmt from an sbt plugin because of the scalameta dependency. Until they have removed sbt plugin they had a workaround using a custom ivy configuration and instantiating needed classes via reflection. I have not looked into it in detail, but could be something, that might help to get it integrated into the build pipeline more seamlessly.

[jonas]

Thanks for the pointers @2m. I briefly looked at scalafmt and its sbt plugin during prototyping but decided it was more fun to play with scalameta and figure out how to deal with running it later.

[olafurpg]

@jonas It seems to be possible to use sbt 0.13.13 synthetic projects to run 2.11/2.12 code in sbt plugins. It's quite simple as long as you don't pass around too many parameters from the build. See https://github.com/olafurpg/scalafmt/issues/597#issuecomment-263054478 for an example implementation.

[jonas]

@olafurpg Thanks, I will give it a try.