JDK9 support, port to new doclet API

[Abnaxos]

JDK9 introduced a new doclet API. First research showed, that the MarkdownDoclet is still doable with the new API. It even looks like it actually makes things easier. I just took a quick look, but it looks good.

I even found the following statement in https://docs.oracle.com/javase/9/docs/specs/doc-comment-spec.html:

Other doclets may accept the same syntax as the standard doclet, or they may support an alternate syntax.

I don't remember having read such a thing before, I thought what MDDoclet does (changing the input syntax), was plain and simple illegal. Well, it isn't, at least not anymore. :)

[perlun]

@Abnaxos Java 10 and 11 have both been released as of now. :smile: Do you have any idea about when you anticipate for this issue to be fixed?

(I noted this when trying to build our javadocs locally. We are still using Java 8 on our CI server, but I am running Java 11 locally for most of my development.)

[pieterbos]

With JDK10, this doclet no longer works due to the old API still being used, at least not when the -d option is used by the gradle plugin.

[manuel-mauky]

Is there any workaround to use this doclet with gradle and JDK 11+?

[sgrimm-sg]

This causes the IntelliJ plugin to stop working in IntelliJ 2019.2 which uses Java 11, and just having the plugin installed causes failures in IntelliJ's documentation viewer.

[tsjensen]

@Abnaxos Please advise.

As it stands today, we will have to migrate our teams away from your plugin for Java 11. 😢

[perlun]

@Abnaxos We are interested in seeing this resolved, and might be able to put some hours into it, to eventually move away from Java 8 for our Javadoc generation. Would you be willing to accept pull requests fixing this to work on Java 9+?

[marco-brandizi]

@Abnaxos, any news on this? Is this project still maintained? I see commits in 2018 that mention the migration to the new Doclet library, but there is no Maven-accessible release and I get errors when I try to build the snapshot for this project (with the provided gradlew).

Thanks in advance for any help.

[Abnaxos]

Is this project still maintained?

Unfortunately, I have to be realistic and say no, it isn't maintained anymore. The reason is this issue right here. It requires a complete rewrite of everything, the new API has absolutely nothing in common with the old one. Therefore, I stopped using it myself, because I of course also moved on to Java 11 and later, without having time to rewrite the markdown doclet.

I may pick it up at some point again, but let's be realistic here: I won't. :(

If anyone wants to take over, feel free to contact me for help. I'd also be happy to announce such a new fork here.

[tsjensen]

Thanks for telling us straight @Abnaxos. Let's hope someone picks up the ball!

[mnlipp]

I forked this project some time ago, initially in order to be able to switch markdown processors.

I'm still using markdown for my comments. As Java 8 is slowly phasing out, and I don't want to keep up Java 8 compatibility only for generating the documentation, I have rewritten the doclet. This turned out to be a complete rewrite indeed, as @Abnaxos has already observed.

I haven't re-integrated syntax highlighting yet and I still have to re-add PlantUML support. But apart from that the doclet is working as can be seen here.

Feel free to build it yourself for testing or use the snapshot on maven. Unless a major issue comes up with the PlantUML taglet, I expect to have finished the new version in a few weeks.

[marco-brandizi]

Thank you so much, @mnlipp (and @Abnaxos too for the work done before)! I'll try that next days.

[mnlipp]

Feel free to build it yourself for testing or use the snapshot on maven. Unless a major issue comes up with the PlantUML taglet, I expect to have finished the new version in a few weeks.

I have released jdk9-API based mdoclet 2.0.0 and PlantUML taglet 2.0.0.

A somewhat larger project using both can be found here.

[marco-brandizi]

Your documentation says:

Maven. Still using this? Well, you’re on your own…

I've tried to fit into Maven, doesn't work:

[ERROR] java.lang.ClassCastException: class org.jdrupes.mdoclet.DocCommentTreeWrapper cannot be cast to class com.sun.tools.javac.tree.DCTree$DCDocComment (org.jdrupes.mdoclet.DocCommentTreeWrapper is in unnamed module of loader java.net.URLClassLoader @91161c7; com.sun.tools.javac.tree.DCTree$DCDocComment is in module jdk.compiler of loader 'app')

(yes, with the --add-exports option).

Moreover, Gradle is no superior, Maven isn't outdated, and unfortunately, I don't have much time to try to fix these problems. Thanks, anyway.

[mnlipp]

[ERROR] java.lang.ClassCastException: class org.jdrupes.mdoclet.DocCommentTreeWrapper cannot be cast to class com.sun.tools.javac.tree.DCTree$DCDocComment (org.jdrupes.mdoclet.DocCommentTreeWrapper is in unnamed module of loader java.net.URLClassLoader @91161c7; com.sun.tools.javac.tree.DCTree$DCDocComment is in module jdk.compiler of loader 'app')

Which means that maven didn't start javadoc in a new JVM with jdk.javadoc/jdk.javadoc.internal.tool exported to unnamed modules. There isn't any other solution because the standard doclet makes every attempt to prevent any "extensions" (including not properly using the API) -- don't ask me why. (E.g. the StandardDoclet class is nothing but a wrapper designed to make it impossible to use the HtmlDoclet as a base class.)

The way to do this with maven seems to be something as shown here. Most likely, you simply need <additionalJOption>-J--add-exports=jdk.javadoc/jdk.javadoc.internal.tool=ALL-UNNAMED</additionalJOption>. But don't ask me, maven is simply not my cup of tea. Either somebody who uses maven contributes the required configuration information or the remark will remain.

[marco-brandizi]

Which means that maven didn't start javadoc in a new JVM with jdk.javadoc/jdk.javadoc.internal.tool exported to unnamed modules.

Unfortunately, that's not the case, I set it with JAVA_TOOL_OPTIONS, the JVM never misses this.

The way to do this with maven seems to be something as shown here.

I could figure out how to invoke the Javadoc tool on my own. The problem is it doesn't work and I've to accept that you don't want to support a build used by tons of projects and developers. Maybe one day I'll find time to make it work myself, can't put that on my schedule at the moment.

But don't ask me, maven is simply not my cup of tea. Either somebody who uses maven contributes the required configuration information or the remark will remain.

I remember that article: Maven is a classic contextual tool: it is opinionated, rigid, generic, and dogmatic

this is precisely why Maven is useful to enforce a clear pattern on a project of tens of developers, and to prevent that any fiddler around defines his/her own, potentially messed-up or not understandable, build cycle.

I understand people don't have to manage projects of tens of developers, or they prefer a different approach anyway, yet it would be nicer if you rephrase "still using it?".

Thanks anyway, for publishing that software and answering these comments.

[perlun]

@marco-brandizi

Unfortunately, that's not the case, I set it with JAVA_TOOL_OPTIONS, the JVM never misses this.

Are you 100% sure this option actually works in this case? The link you refer to specifically mentions that the JVM does indeed ignore this option in some cases:

In some cases this option is disabled for security reasons (for example, on Oracle Solaris operating system this option is disabled when the effective user or group ID differs from the real ID).

---

As for the "Maven vs Gradle" debate, there are probably good points for using both tools. For me (working at a company where we moved to Gradle before I joined a coupled of years ago), the XML-based approach used by Maven definitely feels more "old-school" than the Groovy/DSL-based approach. But I've only used Maven marginally, so I cannot really comment about it being good or bad.

I'm not a native English speaker, but I don't think the "Still using Maven? Well, you're on your own..." comment is that bad, to be honest. @mnlipp obviously is not that interested in supporting Maven and we can't really blame him for that. Why should we? It's his own time, he can do whatever he wants with it and I'm happy and thankful that he's contributed https://github.com/mnlipp/jdrupes-mdoclet as FOSS software to begin with (even though I think AGPL is perhaps a bit overly restrictive, but that decision is also ultimately his choice).

I'm quite certain that if someone steps up and says "hey, I want to make this project work great with Maven", Michael is more than happy to accept PR:s on how to make this happen/improving docs/etc. :+1: That's really all we can demand, IMO.