search

MangelMaxime / Nacara

https://mangelmaxime.github.io/Nacara/
Other
48 stars 7 forks source link

Create the API generator tool #123

Open MangelMaxime opened 3 years ago

MangelMaxime commented 3 years ago

Here is what API documentation could looks like:

For DU:

image

For record:

image

MangelMaxime commented 3 years ago

It seems like I should be able to generate a really rough API documentation generator.

I am able to generate this for all the records in Nacare.Core/Type.fs

image

This is generated from the typed AST provided by FCS which means that I can also have access to TypeInference to documentation function, methods, etc. in the future.

The first step is to be able to generate the documentation as shown above for the DUs and Record coming from Nacara.Core/Type.fs. This will serve as a POC and also will allow me to release Nacara in v1 with a the core API documented.

MangelMaxime commented 3 years ago
TheAngryByrd commented 3 years ago

Hey 👋

I ended up taking the tool for a spin and here are my thoughts:

Tutorial

I know, I know, it's a beta and such, but can always be better. I think it would be good to include a tutorial from install to setup/integration with a new nacara site. I found myself hitting edge cases with how menu.json/nacara.js.config, the base-url of the tool interact, and where to exactly output the files.

API tool needs to know each project

While it makes sense that you need an entry point to start generating docs from, if you have multiple libraries, it might be nicer to be able to pass in a glob or a list and let the tool generate from them all.

API tool should have a watch mode (and maybe handle the publish step for you)

Going from XML comment -> to website involves a change to docs, a publish of that library, then a run of the api tool. It would be a smoother experience if the tool handled taking in the projects it needs to rebuild, publishing them on changes, and generating the markdown files from those changes. See here for an example of that flow. If nacara doesn't want to run dotnet publish, it should at least watch a directory where the dlls/xml get dropped.

Api Tool assumes third party dependencies to be on microsoft docs referencable.

For example: https://docs.microsoft.com/en-gb/dotnet/api/newtonsoft.json.linq.jtoken. This is probably impossible to figure out where the actual docs live for any given third party library but it might be worth either checking for 404s of the site or possibly having a lookup for known packages to known good sites.

Certain tags in the XML aren't displayed

Aren't rendered on the webpage.

API Generation doesn't integrate nicely with the side-nav-bar

While regular documentation has a nice sidebar experience, the API does not. Fake does have a sidebar experience and would be nice to replicate it here since we already have this UX pattern.

Knowing about certain attributes

It would be nice to know about RequireQualifiedAccess and Obsolete (possibly others) and render these somehow on the page so you know about them.

References

MangelMaxime commented 3 years ago

Hey wave

I ended up taking the tool for a spin and here are my thoughts:

Hello thank you for taking the time of looking into Nacara API gen even if it is in really rough state.

Tutorial

I know, I know, it's a beta and such, but can always be better. I think it would be good to include a tutorial from install to setup/integration with a new nacara site. I found myself hitting edge cases with how menu.json/nacara.js.config, the base-url of the tool interact, and where to exactly output the files.

Hum, depending on what the edge cases about menu.json or nacara.config.js are they should be covered in the documentation.

About the generated structure of the files indeed this is something to improve. As I mentioned the API gen is really in early stage, I published it only because I needed to generate the Nacara API instead of writing it manually.

If you think something is unclear in the Nacara documentation don't hesitate to tell me or send a PR.

API tool needs to know each project

While it makes sense that you need an entry point to start generating docs from, if you have multiple libraries, it might be nicer to be able to pass in a glob or a list and let the tool generate from them all.

This is complex to do because F# Formatting doesn't provide the real raw XML comment... That's why for now, I really on dotnet publish to have access to the dlls and the raw xml file so I can parse it.

I need to experiment more to see if it is possible to start a single process to watch all the libraries or if I have to spawn one instance per library.

API tool should have a watch mode (and maybe handle the publish step for you)

Going from XML comment -> to website involves a change to docs, a publish of that library, then a run of the api tool. It would be a smoother experience if the tool handled taking in the projects it needs to rebuild, publishing them on changes, and generating the markdown files from those changes. See here for an example of that flow. If nacara doesn't want to run dotnet publish, it should at least watch a directory where the dlls/xml get dropped.

I agree that it should have a watch mode, it should be improved in future version with a better integration into Nacara config file too.

I would love to be able to kind of generate the documentation of all the APIs of the project at once even for the dependencies to avoid invalid links etc.

But for that, I need to find a way to get the raw comments. Either by finding the XML files or by forking/improving F# formatting to give me access to them.

Api Tool assumes third party dependencies to be on microsoft docs referencable.

For example: https://docs.microsoft.com/en-gb/dotnet/api/newtonsoft.json.linq.jtoken. This is probably impossible to figure out where the actual docs live for any given third party library but it might be worth either checking for 404s of the site or possibly having a lookup for known packages to known good sites.

I was going to write that this a limitation of F# formatting which is true because that part of the documentation if generated by it right now.

But, while re-reading my comment: https://github.com/fsprojects/FSharp.Formatting/issues/708#issuecomment-941009693

It seems like I had found a solution to get around it. I will have a look at it.

Certain tags in the XML aren't displayed

* [example](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/xmldoc/recommended-tags#example)

* [exception](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/xmldoc/recommended-tags#exception)

* [see](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/xmldoc/recommended-tags#see)

Aren't rendered on the webpage.

The example tag needs to be between remark or summary tag and perhaps the returns too.

This is because, I based myself on the C# XML recommendation examples.

The see tag has the safe limitation currently, even if the C# XML recommendation is strange. This is the only tag documented as This tag allows a link to be specified within text. where it is not inside another tag element in their example...

The current parser force the order of the elements like that

  1. Summary
  2. Remarks (not done yet)
  3. Parameters list
  4. Returns
  5. Exception (not done yet)
  6. TypesParams (not done yet) + resolved generics types
  7. See also (not done yet)

The difficulty, is when something is not put inside of these top levels tags as recommanded by the C# XML "specification" right now it get skipped. One way, to solve it could be to make a second pass which removed all the the tags I listed above and if it found something, it happen it to the remarks section. This can still cause strange result if the user write his doc comments "poorly" but at least it will not strip content.

API Generation doesn't integrate nicely with the side-nav-bar

While regular documentation has a nice sidebar experience, the API does not. Fake does have a sidebar experience and would be nice to replicate it here since we already have this UX pattern.

I do agree, this is something that will be hard to do correctly.

The menu is kind of strongly related to which renderer is used for Nacara. For example, if I have a different style for the menu rendered than on the others pages it will looks strange.

Also in case, you missed it there are some anchor generated on the API page

demo_anchor_api

Knowing about certain attributes

It would be nice to know about RequireQualifiedAccess and Obsolete (possibly others) and render these somehow on the page so you know about them.

I agree with you, this is something I left out of the initial really but I need to add it.

References

* [Repo I was using to test with](https://github.com/TheAngryByrd/nacara-tests)
MangelMaxime commented 3 years ago

API Generation doesn't integrate nicely with the side-nav-bar

While regular documentation has a nice sidebar experience, the API does not. Fake does have a sidebar experience and would be nice to replicate it here since we already have this UX pattern.

I do agree, this is something that will be hard to do correctly.

The menu is kind of strongly related to which renderer is used for Nacara. For example, if I have a different style for the menu rendered than on the others pages it will looks strange.

Also in case, you missed it there are some anchor generated on the API page

Actually, the menu are auto generated by Nacara layout so in theory, if I generate h2 elements it should be possible to hook the menu "generator" to it.

dsyme commented 3 years ago

Hey, I'd love to help ensure that the ApiDocsModel coming out of FSharp.Formatting is sufficient - I know there are issues like https://github.com/fsprojects/FSharp.Formatting/issues/708 and I'd like to understand what you need.

The visuals of your API Docs are also great and maybe it's feasible to fold that back into FSharp.Formatting too once mature, as an optional replacement for GenerateHtml.fs

MangelMaxime commented 3 years ago

Hello,

thank you for looking at the work done with Nacara :)

I am still experimenting so I am unsure if my list will be exhaustive but what I need right now is:

  1. Access the to Raw XML comments: https://github.com/fsprojects/FSharp.Formatting/issues/710

    This is needed to apply the comment formatter included in Nacara which is an improvement over the one from Ionide

    Right now, I am hacking my way into the XML file and using regex to try find the XML comments.

  2. Access to the type information to generate the code section at the top of the files.

image

This code, is not coming from the F# source but reconstructed from the type information coming from FCS from objects like FSharpType or Symbol etc. This allow me to generates links between the types and also apply syntax highlighting.

The code is not pretty but I think this can gives you an idea of how I am generating the snippets.

https://github.com/MangelMaxime/Nacara/blob/33dc6c6f38d4f992617f15a8cb20fc9b9e3a88d4/src/Nacara.ApiGen/Source/Render.fs#L223-L370

  1. Bonus: Have F# formatting works "out of the box"

What I mean by that, is that currently I need to run dotnet publish and then run F# Formatting against the generated folder. This is the easiest way I found to have the all DLLs information.

I understand that doing the DLLs/dependencies resolution is a complex thing but the IDEs are already doing it. So it would be nice if F# formatting could do the same.

Doing dotnet publish right now also have the benefit of providing me the XML files because of this issue https://github.com/fsprojects/FSharp.Formatting/issues/710

The visuals of your API Docs are also great and maybe it's feasible to fold that back into FSharp.Formatting too once mature, as an optional replacement for GenerateHtml.fs

Thank you :)

It would be nice indeed, it would be nice especially to benefit from features like watch mode instead of doing a full compilation of the library at each changes which takes some times to do.

The only thing is that Nacara needs to have a Markdown file to process with header at the top.

Example:

---
layout: api
---

When I have a stable layout and customisation system for the API layout. I would be happy to ping you to discuss about adding it back in F# Formatting if that still makes sense.

PS: I know that you are working on fixing some of the issues I reported at the same time I write this message.