Generate ExDoc documentation for Erlang projects

[erszcz]

This thread documents the progress on generating ExDoc documentation for Erlang projects.

Current status

erszcz/edoc@0c81cea1b659ad9fd00993d25c9de9293c3651b0 can be used to generate chunks compatible with erlang/otp@dd3b0157d2b20adacd98fea64b3af443b6639b9a as follows:

git clone https://github.com/ferd/recon
cd recon
cat >> rebar.config <<END

{plugins,
 [
  {rebar3_edoc_chunks, {git, "https://github.com/erszcz/edoc.git", {branch, "master"}}}
 ]}.

{provider_hooks,
 [
  {post, [{compile, {edoc_chunks, compile}}]}
 ]}.
END
rebar3 compile
ls _build/default/lib/recon/doc/chunks/

ToDo

• [x] Aim for 100% compatibility with application/erlang+html format.
• [x] Define a set of HTML tags which are allowed in the chunks (likely starting with tags from https://developer.mozilla.org/en-US/docs/Web/HTML/Element but removing those which are obviously not useful in an .erl file / non-dynamic webpage).
• [x] Make sure EDoc documentation discourages use of raw HTML in doc comments.
• [x] Make sure that links to modules/functions/types/callbacks are stored properly.
• [x] Include since and deprecated information in the chunks:
  • [x] @since
  • [x] @deprecated - partially supported, see https://github.com/erszcz/edoc/commit/ed5f585ad9b3d7ed138cef0b897ecca113861ba7. EDoc @deprecated might contain markup, but EEP-48 expects a flat binary() for this metadata field - currently, all markup is dropped in the chunks.
• [x] Remove temporary edoc_layout_chunk_markdown layout and refactor the rest for simplicity:
  • [x] Remove edoc_layout_chunk_markdown.
  • [x] Refactor.
• [x] Prepare a PR to erlang/otp - this will likely squash the forked project history.
• [x] Once the PR is created, If need be also fix NEW-OPTIONS: / INHERIT-OPTIONS: doc comments - I'm not sure how to use these yet, though.
• [x] From 2020-03-10 WG meeting: have a command line tool to generate the chunks. The escript doesn't yet handle -I, i.e. adding custom include paths, but it hasn't yet occurred to be a problem.

2020-04-01 update:

• [x] tt, expr, see, strong tag handling - these pop up in Recon and EDoc itself
• [x] ex_doc consuming doc chunks could not generate EDoc documentation due to the types exported to chunk and types exported to Dbgi not matching. This is solved by rewriting all @spec / @type tags to -spec/-type attributes (currently only for edoc.erl)
• [x] HTML generated by ex_doc for edoc.erl (after spec/type rewrite) has some documentation entries truncated, e.g. for the deprecated file/2

[jfacorro]

Hi 👋 ! This looks like a great idea, I was wondering what is the current status of this effort.

I was also curious about how would an Erlang project use ExDoc in a purely Erlang context (i.e. Elixir is not installed). I guess Elixir would be required in that case, am I correct?

[josevalim]

We are working on it. At the moment, we are refactoring ExDoc so it can support multiple sources (Elixir's markdown and Erlang's doc AST).

I was also curious about how would an Erlang project use ExDoc in a purely Erlang context (i.e. Elixir is not installed).

It will be done with escripts - so Elixir, ExDoc, and everything else will be in a single file executable.

[erszcz]

Hi, @jfacorro!

On the EDoc side of things the remaining big thing is to support linking. The progress has been slow lately due to the summer period and the pandemic before that, but I'm now hoping to get up to speed again.

BTW, funny things are exposed when trying to fill in the chunk fields with what we can get from EDoc - for example, @since or @deprecated were never supported on types, whereas prior to this effort callbacks could not have attached doc comments at all 🤷

[erszcz]

FYI, the PR for the EDoc side of things is now ready - https://github.com/erlang/otp/pull/2803.