Open pitag-ha opened 2 years ago
I know this is an old issue and most things in the list have been done, but as someone who decided to write their first ppx a while ago, I have to say that the manual is nice but there could be a section dedicated to the parsetree (or maybe improving the ast builder/pattern documentation) as it's very hard to find any info about the ast for beginners. I wouldn't mind doing that myself but then again, I don't really know everything about the ast and there's not a lot of information online.
Hi, I'm a new learner of ppxlib, and here are some possible improvements I found about documentation. This issue was originally discussed in a post in discuss.ocaml.org.
Here, I'll use the documentation of Ast_helper.Exp
as an example, but I found this to be quite common in the API part of the documentation.
I'd say that the doc does a good job listing all the APIs, including the type of arguments and return values. However, more explanation and examples would be really helpful.
For example, a short explanation can be added at the top of the page stating that the purpose of Ast_helper.Exp
is to construct expressions in the AST, and some code examples with comments like this can be added (the code came from a modification of a tutorial written by @patricoferris in this post):
| { ptyp_desc = Ptyp_constr (lid, args); _ } ->
(* We generate the name with "_stringify" on the end *)
let new_name = Expansion_helpers.mangle_lid (Suffix "stringify") lid.txt in
let new_lid = { lid with txt = new_name } in
(* We turn it into an identifier hoping we have already generated
the function either by hand or by using the ppx *)
let fn = Exp.ident new_lid in
(* We construct an application of this function to an argument.
Note that we assume the function may be higher-order if the type
contains parameters e.g. int list *)
let app =
if args = [] then fn else
Exp.apply fn (List.map (fun x -> Nolabel, expr_of_type x) args)
in
[%expr fun x -> [%e app] x]
Also, I very much agreed with @ComanderP's opinion on adding a specific page to explain the overall design idea or a bigger picture of the AST. For example, I was able to grasp most of the meaning of the AST by looking at this page, but one thing that resulted in some confusions was the naming convention of the Astlib
.
For example, some things are named like Ptyp_any
, and others are named like ptype_name
. Without explanations, I can't really tell what the consideration is for making "typ" and "type" different. Another example is naming like pcd_name
, and it is hard for me to tell what does pcd
stands for. If we can add some explanations for the naming convention of Ast_lib
, it will definitely be helpful on the code quality of ppx extension developed by beginners like me, as they can make sure the naming in their extension is aligned to the standard (both ppxlib and the OCaml AST).
Current situation
As usual, we have different kinds of docs: a ~manual~ (outdated), the API documentation with integrated manual, a section about metaprogramming on ocaml.org, a couple of very simple examples, the dune docs on preprocessing specifications. Apart from that, very important resources to learn about how to write a ppx rewriter are a blog post and the compiler's parsing/parsetree.mli file.
The only resource that's kind of complete to learn how to write an extension node rewriter or a deriver is the blog post. The "manual" is more of a very high-level introduction to PPX (which is also very valuable but serves a different purpose) than a detailed manual. The API docs miss docs for some important values and aren't rendered perfectly. Apart from the docs for values/types, the API docs also include some important information about some of the modules (e.g.
Ast_builder
,Ast_pattern
,Ast_traverse
), which isn't easy to find and difficult to put into context.How to improve the situation
For the resources that make sense to co-exist separately, we should make them all easy to find:
Apart from those resources, we're currently also strongly relying on the blog post. Instead, we should integrate everything relevant in there (that is, most of it) into our manual and also write up some new bits for our manual:
Ast_builder
andmetaquot
. Formetaquot
we already have something in the manual.Ast_pattern
andmetaquot
. AboutAst_pattern
: To explain the general concept ofAst_pattern
, we need to write something new; to explain the type parameters inAst_pattern.t
, we can re-use a blog post section; to explain its combinators, we can re-use a discuss answer and link to the module docs.And in our API docs, there are also some things that could be improved:
Ppxlib__.Import.<some_type>
types around in our docs (which lead to the situation that folks would use our internalPpxlib__.Import
API), which now I'm not seeing anymore. We should find out if that was fixed by odoc or if I'm just missing it now.Ocaml_shadow
include in the library interface moduleOf_ocaml
) module allows AST nodes to be converted between the AST version used by ppxlib and the AST version used by the compiler the project is compiled with. The latter is variable and depends on the project. So, if that isn't much work, it would be nice to represent that version in our docs by a place holder such as<compiler_version>
:Currently that version gets fixed by the compiler version odoc uses when generating the docs (apparently, 4.05):
How to priorities
That's a long list of tasks, but lots of them only consist in moving info and adding links. Apart from that, I'd prioritize the new manual sections and adding missing API docs for the most important values. We can then complement the API docs for the more advanced/rare features over time.