Interop with Docusaurus

[aantron]

Basically, odoc would be parsing the docs and doing the cross-referencing, and Docusaurus would be doing the displaying.

Docusaurus is already in use by reasonml.github.io and bucklescript.github.io, and seems to be doing a good job there.

cc @kennetpostigo @bikallem

[bikallem]

This is cool. Docusaurus seems to require markdown files. Does odoc produce markdown?

[bikallem]

Answering myself. It doesn't.

[Khady]

It is a bit annoying to have to install one more package manager and use npm/yarn to generate documentation. But it guess we can live with it...

[dbuenzli]

I'm not sure I understand. What does Docusaurus provide ? If it's just about the rendering then odoc can simply inspire itself for its markup structure/css.

[bikallem]

From what I understand, https://docusaurus.io/ provides the facility to generate whole documentation website including blogs and API docs - similar to https://reasonml.github.io/ and https://bucklescript.github.io. This is probably useful for those without much website design/development skills.

[aantron]

@Khady Docusaurus wouldn't be a requirement of odoc, just an option. Outputting Markdown for Docusaurus might be more maintainable and produce much higher quality in the short term than figuring out odoc's markup.

[jordwalke]

Docusaurus can also accept ReactJS components and render them. And actually, for syntax highlighted code examples (or type signatures) with clickable links (to jump to type definitions etc), markdown isn't even sufficient. I was thinking perhaps we could agree on a standard convention - almost a superset of markdown, which various tools including Docusaurus could interpret - which would be suitable for highlightable source code examples with clickable links, as well as API reference documentation with clickable links. Tools like Docusaurus could take this markdown superset, extract meaning from it and render it beautifully including generating the reference list on the side navigation etc. There's reasons we'd want to integrate with Docusaurus as opposed to being something completely separate - we want the site wide search to integrate nicely. If we come up with a standard convention, no one would be forced to use Docusaurus if they had reasons not to.

[jordwalke]

cc @JoelMarcey who made Docusaurus and continues to drive its improvements. I believe navigable source code documentation and API references is one place where markdown is lacking, so we might want to collaborate on extending it.

[JoelMarcey]

I believe navigable source code documentation and API references is one place where markdown is lacking, so we might want to collaborate on extending it.

@yangshun Isn't this something we are considering for v2?

[MoOx]

I would like to contribute to this discussion by bringing my 2 cents. Sorry if my story is long, but I prefer to give too much infos instead of not enough.

Context: I am coming on this issue because I wanted to improve bs-react-native docs which is currently not really pretty and convenient to browse, and since I made a static site generator (which is friend with react & reasonml), I am looking if odoc could produce alternative outputs (like a low level format instead of html or markdown or a superset - more about this below).

---

I am currently developing phenomic, a static site generator generic enough to not be tied to markdown (or anything actually), with react as a renderer (even if others solutions could be implemented - currently, since I use react daily, I did only this one).

When files are "send" to phenomic (which is optional, see later), the inputs (markdown, asciidoc, json) are converted to... just json (and injected into a "kindof" database so you can retrieve this data during the "website compilation" step). Given this and the fact that phenomic can consume data from any source (like an app, à la next.js static getInitialProps) it make sense imo to work on a low level output that could be used from any kind of language/framework/whatever.

To be more specific about how phenomic handle markdown: I transform markdown to a "react ready format" so I can easily render this

• Input https://github.com/phenomic/phenomic/blob/b01188b26dd6eaee8a0740ccd1041faa0649033b/packages/helpers-transform/src/__tests__/unifiedProcessor.js#L19-L32
• Output https://github.com/phenomic/phenomic/blob/b01188b26dd6eaee8a0740ccd1041faa0649033b/packages/helpers-transform/src/__tests__/__snapshots__/unifiedProcessor.js.snap#L32-L358

Output is easily rendered with a react component

• ReasonReact version: https://github.com/phenomic/phenomic/blob/b01188b26dd6eaee8a0740ccd1041faa0649033b/packages/plugin-renderer-react/src/bodyRenderer.re
• react js version https://github.com/phenomic/phenomic/blob/b01188b26dd6eaee8a0740ccd1041faa0649033b/packages/plugin-renderer-react/src/components/BodyRenderer.js

Note 1: in the example, code as already been "highlighted" in the output and if rendered using the given react component & the proper necessary styles (just a highlight theme) it looks nice Note 2: the "bodyRenderer" which take json and output "html" can accept a map of components so you can render anything to anything (eg: choose that t: "h2" are rendered using <Title level=2>, t: "video" using another given react component etc) Note 3: implementation of the transform step is currently slow as it rely on many js packages (unified, remark, rehype) but I plan to change that

My idea with phenomic is to convert as many as possible code (in the coming month) to reasonml so I can run part of it in native code, for obvious performance reasons (anyone that tried gatsby or next know how slow the developer experience can be, especially compared to bsb) and benefit of fastpack to finish my evil plan to conquer the current hype about site generator allowed to use react as a renderer on the server (pre-rendering) or the client.

Also another win for json is that (think about "front-end") it's very easy to serve json. Phenomic helps to pre-render your site as pure html, but also to serialize requested data as json so the client side could only retrieve raw json, and offers an app like experience.

---

@jordwalke note about a superset of markdown: I am not sure if that approach is the right one. We could think about mdx, a mix of markdown & jsx (in short you can add js and "return react components from blocks" in addition to classic mardown) but I am pretty sure that current implementation cannot be fast or easy to maintain/evolve outside md+js.

---

I wanted to share with you this usage because it's think it would be a good idea to consider an easy to render low-level json "ast" as an output source. Any other tools could very easily retransform this output as anything they want. If you produce json (instead of html) in a format similar to what I showed (t: tags, p: props/html attributes, c: children) anyone could easily reconvert this ast to markdown or whatever Docusaurus or other tools would like to consume.

[giltho]

Hi ! I'm also planning to write a docusaurus website. The solution I see for now is simply linking the odoc-generated html as static files. It would however be nice to have odoc generating md files. What are the limitations right now ?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[JoelMarcey]

The solution I see for now is simply linking the odoc-generated html as static files.

That is what a lot of people do with v1 of Docusaurus and it works pretty seamlessly with the generated site.

But with Docusaurus' v2 plugin architecture, a plugin could be made to do this as more of a first-class component instead of generating html totally separately outside of the Docusaurus infra.

Is this still something people would like to see?

cc @yangshun

[aantron]

From the maintainers' side, we would like to see it, but we won't be able to get to it soon, so we had the stale bot close it to keep the tracker focused. In fact, we've since changed the stale message to clarify that we will periodically look at issues closed with stale, to see if they should be resurrected:

https://github.com/ocaml/odoc/blob/8f9f5294762b9910c195eb148ef12b54b78afc9b/.github/workflows/stale.yml#L16-L31

And, as it says, further discussion is still welcome even if the issue is closed :)

[giltho]

@JoelMarcey Using docusaurus v2, I have been blocked by the internal router and unable to link the statically generated documentation. If I add the whole documentation to the static assets and put a link to it, it is detected as an "internal link" and I'll get a nice 404 page because it's not a registered page. But if I go to the url directly by entering it in my browser, it finds it. That being said, it was on 2.0.0-alpha.48 so 6 alpha versions ago, I need to check again. If this is still true, it looked like adding odoc-generated content to docusaurus v2 is just impossible for now.

[mbarbin]

Thanks for this issue and the conversation. I'm interested in the Interop with Docusaurus, and created a sample project to explore it in practice : https://github.com/mbarbin/doc-experiment-docusaurus

Thanks @giltho, with inspiration from your last response I've indeed resorted to including the odoc-documentation into the static assets for now, and added a link to the url directly, with the aim on improving incrementally if enhancements are available in the future.