search

hoaproject / Central

Hoa is a modular, extensible, and structured set of PHP libraries.
https://hoa-project.net/
Other
104 stars 8 forks source link

Online API browser #53

Closed Hywan closed 6 years ago

Hywan commented 7 years ago

Hello fellow @hoaproject/hoackers and users!

I would like to have a nice online API browser for our libraries. Ready?

Intro

So far, we have tried http://api.hoa-project.net/, which is the result of Doxygen output.

Is it a solution we could maintain? Do we have better alternatives? What do you expect from an API browser?

Solutions for PHP

Personally, I don't like any of them.

Solutions in other languages

Build our own?

I really like the approach of rustdoc, the style, the fact that the code must be extremely documented, and the API is not the first-citizen. Look at Tagua VM, the https://tagua-vm.github.io/parser/ page represents the documentation of the tagua-parser library.

For each class, we have a long description. Then we have modules (namepaces in PHP) with a description.

For each function (for instance parse), we have a description, and an example. The API is even not documented. Well, this discussion is part of the RFC #58.

But the browsing experience is great.

Outro

All API documentation generator I tried in PHP didn't catch me. In other languages, like Rust or Elm, we have very nice ideas. We must get inspired from them.

I am not afraid of building a new tool. I personally thing this is the direction to take.

Thoughts?

vonglasow commented 7 years ago

Have you tried apidoc ?

1e1 commented 7 years ago

Quick and dirty answer: How can it be filled by Praspel and typed functions (new in PHP7)? Can we use Atoum for extracting some examples? I will enjoy if the generator need no more comments.

1e1 commented 7 years ago

Remember the code is the best documentation! Notice: in an online API browser we are mainly looking for public methods.

Hywan commented 7 years ago

@vonglasow APIDoc is for Rest API, no? @1e1 We need to expose all methods, whatever the visibility they have, you don't know who is using your libray and for what. For the rest, I didn't understand what you meant, sorry :-/.

1e1 commented 7 years ago

This API browser would not add extra comments for generating the documentation. There is enough constraints from other tools. It should use these constraints as inputs.

Pierozi commented 7 years ago

As RFC #52 will produce high quality and dynamic content We should be able to extract Markdown from code and generate HTML verison.

Maybe http://www.mkdocs.org could be useful?

Hywan commented 7 years ago

MKDocs looks nice, but I am afraid this is too much oriented for documentation, and not API documentation, aka code-oriented. What do you think? Do you think it will be difficult to bend MKDocs for an API browser?

Hywan commented 7 years ago

Current implementation https://github.com/Hywan/Kitab.

harikt commented 7 years ago

It looks like you love parsing the docblocks. I have came across lithium which does something similar. Not a standalone one. But it has some plugin when installed your whole framework docs all will be shown.

The plugin is : https://github.com/UnionOfRAD/li3_docs

The documentation written in class is as

https://github.com/UnionOfRAD/lithium/blob/5d9a06b8667e20490ef5ccb3275be8824f77666b/action/Controller.php#L15

once it is rendered you will see something as : http://li3.me/docs/api/lithium/1.1.x/lithium/action/Controller .

I guess this is what you are looking for. I looked at Kitab, but as it has less information how the docs should be written etc I am not very sure also.

Hywan commented 7 years ago

@harikt The format is defined in RFC #58. The output is not exactly what we want. We are looking for something closer to what rustdoc does.

Hywan commented 7 years ago

Yesterday progressions

A namespace page

kitab_1

On the left, we have the navigation, which is “all the sibling data”. You must understand it as “namespaces in the same level”.

On the center, we see one namespace in details, with all entities it contains: Classes, interfaces, traits, and functions.

A class page

kitab_2

Today progression

The class page has been enhanced with more data. Markdown is converted into HTML for the doc comment attached to methods. We have syntax highlighting. Markdown to HTML does not auto-convert entity names (like class names) into links yet. This is expected in a close future.

screen shot 2017-03-17 at 10 02 55-fullpage

The class documentation will be placed just before the “Methods” heading.

Hywan commented 7 years ago

Today progression

And now we have a static search engine screen shot 2017-03-21 at 11 12 47

It is kind of slow when we have more than 600 files to analyse, but I am working on a cache (the slow part is to build the search index, we can cache it, but I don't know how to orchestrate that).

Anyway, it works great 🙂. I have decided to write it with Elm, because why not.

Here is the Search.elm module.

Hywan commented 7 years ago

Great news! The search data are now compiled ahead-of-time. What does it mean? 3 files are now computed:

  1. search-database.json, contains all the search information (never used by the client-side, never loaded, only needed for the compilation step, moved to the frontend just for debugging),
  2. search-index.js, contains the serialized and compressed index, it is compiled ahead-of-time by Kitab, the client only unserializes it and here we go,
  3. search-metadata.js, contains metadata useful to print results to the user, like URL, real symbol name, and description.

Having the index and the metadata separated really helps to reduce the size of the index (by a factor of 65% with my recent tests).

Why doing this? To reduce the initialisation of the search on the client-side. Now, initialisation of the search drops from 17s to… 369ms :tada: with an index representing 606 files. Here are more numbers:

Description Kitab only Kitab and its dependencies
Number of PHP files to analyse 29 606
Time to compile everything 0.25s 6.57s
Size of search-database.json 5.4Kb 149Kb
Size of search-index.js 4.4Kb 199Kb
Size of search-metadata.js 3.9Kb 117Kb
Time to initialize the search on client-side ? 369ms

I am very happy with these results. Not to say that search-index.js and search-metadata.js are easily cachable with a 3xx HTTP code. They are purely static.

The size of the search module search.elm.js is 92Kb. So for Kitab only, the client loads 100.3Kb, and for Kitab and its dependencies, the client loads 408Kb, once.

We have a complete reactive UI with an advanced search for 600+ files for 400Kb. This is still less than Ember or Angular alone —just the framework, not the app— (source).

Hywan commented 7 years ago

Interface support for the HTML target are better. Class and interface constants are supported by both the IR and the HTML target (including constant visibility, and default values, for inline or grouped form, resp. const A; const B; or const A, B;), see https://github.com/Hywan/Kitab/commit/ad3a72231c9e997b8be065dfab59cd724fc08e2a.

screen shot 2017-04-26 at 15 29 47

Grummfy commented 7 years ago

Seems really nice. Just a nice feature, for future usage outside Hoa could be a way to +/- compile it to other template style/design

Hywan commented 7 years ago

@Grummfy Sure. This is always the case. Choose another target, and here you go, https://github.com/Hywan/Kitab/blob/ad3a72231c9e997b8be065dfab59cd724fc08e2a/src/Compiler/Compiler.php#L91.

Hywan commented 7 years ago

Progression summary:

  1. Attributes are supported (https://github.com/Hywan/Kitab/commit/9f376273df5854fe873da410011770cf54fae0eb),
  2. Introduce namespace description, see below please (https://github.com/Hywan/Kitab/commit/9474b3499ef76714f6ddfbfc789d3c8cfe52a8ee and https://github.com/Hywan/Kitab/commit/c6b8f3e53bd36ebfd83251ad5d4a8b3f897f9189),
  3. Better global/unnamespaced entities support (https://github.com/Hywan/Kitab/commit/b61fce887931c2434378f345683c70fed25a512f),
  4. Functions are supported (https://github.com/Hywan/Kitab/commit/1c753b10ea5834d0565d40c073c7897ff5f201b8),
  5. Links to source have been added, please see below (https://github.com/Hywan/Kitab/commit/7fa57a11d6f6e773f31544f11af3fdbdc502541b and some others).

Note about namespace description. Any classes, interfaces, traits, or functions can have a block comment attached to it, in a single place. However, for namespaces, we have many locations where to write the documentation. Because this is not obvious and undeterministic, I have decided to look for a README.md file in the directory representing the namespace —with regards to the PSR-4 standard. If such a file exists, then we use it as the namespace description. Thus, I have introduced a new --with-composer option to read the PSR-4 mappings from a composer.json file. Thoughts? My first idea was to add a new special Description.md, but using README.md makes a lot of sense: It describes the content of the current directory, and it is displayed as HTML on Github, and Gitlab, which is very useful too.

About links to source, it is attached to each entities (classes, interfaces, traits, functions, and even methods).

Here is an online example https://hywan.github.io/Kitab/kitab/index.html of https://github.com/Hywan/Kitab. On this documentation homepage, you can see the README.md of the src/ directory. Below that, you have the Classes and Namespaces Sections. I have found a bug just right now, so links to namespaces are broken, but here is the link to the Kitab\Compiler namespace, https://hywan.github.io/Kitab/kitab/compiler/index.html, and here is the documentation for the Kitab\Compiler\Compiler class, https://hywan.github.io/Kitab/kitab/compiler/Compiler.html. You can see the Attributes, and Methods Section. All the [src] links are working. For example, links to the compile method, https://hywan.github.io/Kitab/kitab/compiler/Compiler.source.html#L.91-106. The static search engine is also working.

Please, fill bugs or discuss here :-). Feedbacks are really appreciated!

Grummfy commented 7 years ago

can you add an example where you use php 7 synthaxe with return, scalar type, etc.

Otyherwise nothing special to say it seems really nice

Hywan commented 7 years ago

@Grummfy Like this one https://hywan.github.io/Kitab/kitab/compiler/intermediaterepresentation/Into.html?

Grummfy commented 7 years ago

Well in this case, I would expect that "Type" or "File" would be clickable because thier are class that exist inside the documentation. For that's the only expected things that I see.

As extra information/plugins, I really enjoy to have stuff like phpxref to cross check the usage. But not directly accessible.

Hywan commented 7 years ago

As extra information/plugins, I really enjoy to have stuff like phpxref to cross check the usage. But not directly accessible.

Can you give an example please?

Grummfy commented 7 years ago

http://phpxref.sourceforge.net/phpxref-xaraya/nav.html.gz?index.html.gz

Hywan commented 7 years ago

This a tree explorer, yes, and… ? I don't understand what you are asking :-).

Hywan commented 7 years ago

Bug in namespace links is now fixed, https://hywan.github.io/Kitab/kitab/index.html.

Next steps:

After that, we will be good to try a 1.0 beta, and deploy it on Hoa.

Hywan commented 7 years ago

Live: https://hywan.github.io/Kitab/kitab/compiler/Compiler.html.

screen shot 2017-05-30 at 12 39 04

And here we go, since https://github.com/Hywan/Kitab/commit/1950f62048dd98c82242dffb03fd32fce8993c08, all qualified names are clickable. I still have an issue with extends that are not linkable, a small conflict with Prism (c-like profile).

Hywan commented 7 years ago

Progression summary

I find interesting how to document things without saying: “Return bla bla” or “Get bla bla”, because the name of the function or method must be self-explanatory! So taking the example of inNamespace in https://hywan.github.io/Kitab/kitab/compiler/intermediaterepresentation/Entity.html, I used the following description:

An entity is in a namespace if its fully-qualified name contains at least one namespace separator (aka \).

Actually, this is the only explanation we expect at this step. We know the method is called inNamespace so saying something like: “Check if the name is in a namespace” is totally stupid.

This is interesting to rethink how we write documentation.

Pierozi commented 7 years ago

Really nice, I like the Namespace page related to Readme, a good one :+1:

Hywan commented 7 years ago

It renders on Github too! On Github: https://github.com/Hywan/Kitab/tree/master/src/Compiler, on Kitab: https://hywan.github.io/Kitab/kitab/compiler/index.html.

Hywan commented 7 years ago

And now, Kitab can test your examples with ./bin/kitab test. More at https://github.com/Hywan/Kitab/pull/3.

Hywan commented 7 years ago

What's next?

brzuchal commented 7 years ago

None of the links to generated documentation doesn't work. Could you provide working one?

Hywan commented 7 years ago

Yes, we are migrating the repository into the Hoa organisation since this morning. Give me some hours to fix everything :-).

Hywan commented 6 years ago

@brzuchal https://hywan.github.io/Kitab/kitab/index.html :-)

Hywan commented 6 years ago

The 0.10 version has been tagged https://github.com/hoaproject/Kitab/releases. The version is stable. We use it at work.

It is implemented inside hoa/test. We will start to deploy the documentation on the hoa-project.net site soon.