Removing compiled `docs` from the source code repository

[nirname]

Idea

I think that we should remove docs (compiled version) folder from this repository, and move it to a separate one. Keeping it there equals to having dist in the repository along with source code.

Why it is there

• It was very convenient at the start of the project, easy hosting on Github Pages.

What to do

• Remove compiled docs and put them in a separate github-pages dedicated repository
• Do not forget to switch github pages to this new repository
• Put the folder in gitignore here
• Reconfigure vitepress if needed
• Then we also need to remove this warning
• Configure CI to build and compile docs automatically

Reasons for deletion / moving

• The less branching in instructions the less room for error
• Fewer files in repo
• 'docs' becomes stateless, you can easily remove it, alike node_modules
• No need to setup different folders for development and production in vitepress, simpler configuration, simply put everything that is compiled to docs folder
• Easier to search in editor and repository on github, no need to exclude docs folder while searching, no confusion
• Bot will not commit to your branch, cleaner history
• No need to tell every docs developer to enable workflows on forks to turn this bot on

Originally posted by @nirname in https://github.com/mermaid-js/mermaid/issues/4382#issuecomment-1622653453

[aloisklink]

I think that we should remove docs (compiled version) folder from this repository.

I'm :+1: for completely removing it too. Or even better, moving packages/mermaid/src/docs into the docs folder.

I think the main reason why we haven't is that it's because if we have a Mermaid code block like:

flowchart TD
    A[Hello] --> B(World)

GitHub will render the Mermaid diagram, but there's no way to see the Mermaid source-code for a diagram. So my gut feeling is to wait and hope GitHub adds this feature.

Originally posted by @aloisklink in https://github.com/mermaid-js/mermaid/issues/4382#issuecomment-1622944432

---

@aloisklink I do not fully understand how is it related to Github mermaid preview, could you elaborate more on that? Does anyone use github for searching for compiled docs? There is https://mermaid.js.org/ for that... I am not asking to move docs to separate repository comletely, only a compiled version of it

Originally posted by @nirname in https://github.com/mermaid-js/mermaid/issues/4382#issuecomment-1623410616

Ah, sorry, the docs at docs/ are NOT needed for GitHub Pages. Mermaid uses the actions/deploy-pages action to upload docs to GitHub Pages instead, see:

https://github.com/mermaid-js/mermaid/blob/194ef202ac5df51056d98c14f13e9ca85f274c35/.github/workflows/publish-docs.yml#L57-L59

I believe the main reason why we need to compile the docs from packages/mermaid/src/docs to docs/ is because we want to show both the Mermaid source-code for a diagram, and the rendered Mermaid diagram (using GitHub's Mermaid feature):

Mermaid can render Pie Chart diagrams.

pie title Pets adopted by volunteers
    "Dogs" : 386
    "Cats" : 85
    "Rats" : 15

pie title Pets adopted by volunteers
    "Dogs" : 386
    "Cats" : 85
   "Rats" : 15

Because GitHub's Mermaid preview only shows the diagram (there's no easy way to see the source-code of the Mermaid diagram), we need to duplicate the Mermaid diagram definition, once with a ```mermaid-example (so GitHub shows the code instead of rendering the diagram), and once with a ```mermaid (which GitHub renders):

https://github.com/mermaid-js/mermaid/blob/194ef202ac5df51056d98c14f13e9ca85f274c35/docs/syntax/pie.md?plain=1#L12-L26

I believe this is the main reason why we can't just delete docs/ and replace it with packages/mermaid/src/docs. If GitHub ever adds the feature so that you can both view the rendered Mermaid diagram and view the source code of a Mermaid diagram from a single ```mermaid code-block, we can probably remove these compiled docs, since the source docs at packages/mermaid/src/docs will be almost exactly the same.

Does anyone use github for searching for compiled docs? There is https://mermaid.js.org/ for that...

We do get quite a few PRs that only edit the files in docs/ instead of packages/mermaid/src/docs, despite the mermaid.js.org website pointing to packages/mermaid/src/docs, so maybe :shrug:

From https://github.com/mermaid-js/mermaid/graphs/traffic, apparently mermaid/docs gets about 500 views a week, and that doesn't count the individual pages.

It might just be worth deleting docs/ completely, since I'm guessing then that most people will move to using mermaid.js.org instead of looking at packages/mermaid/src/docs.

[Yokozuna59]

...If GitHub ever adds the feature so that you can both view the rendered Mermaid diagram and view the source code of a Mermaid diagram from a single ```mermaid code-block, we can probably remove these compiled docs, since the source docs at packages/mermaid/src/docs will be almost exactly the same.

Is it because there isn't a perfect way to highlight code or something else? I guess this could be resolved with Langium, just if I knew how to render a diagram without the language server :|

[nirname]

Github pages used to work in such a way, that they could serve a folder from your repository. So probably docs folder was there for this reason initially. May be docs is a "fallback" for the domain, I don't know. But it has to be treated like a separate version of "deploy", anyway. Do we have any stats about https://mermaid.js.org/ so as to compare its RPM's to docs folder on GH? Regarding syntax highlight - there is some (I dont know how it works) on mermad.js.org, there is no one on GH.

[aloisklink]

Do we have any stats about https://mermaid.js.org/ so as to compare its RPM's to docs folder on GH?

It looks like we do!! https://github.com/mermaid-js/mermaid/pull/4473

It looks like the root path (https://mermaid.js.org/) had ~30k views in the last week, which is a lot more than the 500 views that https://github.com/mermaid-js/mermaid/tree/develop/docs had!

With those numbers, I wouldn't mind too much if we deleted docs/ completely, but it's worth getting feedback from the other Mermaid team members, and the community.

Is [the reason why GitHub doesn't show the source-code of Mermaid diagrams in ```mermaid blocks] because there isn't a perfect way to highlight code or something else?

I think it's more that normally, if the average GitHub user writes a ```mermaid code-block, they want to show the reader the rendered Mermaid diagram. Viewing the source-code of the diagram is pretty much only important to us, since we want to show people both the rendered diagram, and the diagram code, so that we can teach people how to use Mermaid.

GitHub did recently add the Copy button to copy the source-code of a Mermaid diagram, but making that change is a lot easier than making a button to show the source-code:

[sidharthv96]

Let me try to make sense of the docs journey.

1. We had docsify which needed .md files.
2. These docs had code + diagram inside them, in 2 separate manual blocks.
3. These blocks went out of sync in many places, causing issues.
4. There was a discussion on removing the code block and just keeping the diagram block, modifying docsify to show both.
5. Someone (I don't remember who) said we should keep the code block, so users on GitHub could see both.
6. I created the docs.mts script to remove the duplicated code blocks.
7. docs.mts grew to add more functionalities.
8. @emersonbottero migrated the docs to vitepress (great job, again :rocket:).
9. Vitepress transformation capacity was added to docs.mts.
10. This is when the generated /docs in the repo lost its importance, as docsify was using it till then.
11. We started having issues with users making changes in the wrong place, even after adding big warning headers and setting up proper links to edit page.
12. I added the docs build pipeline, which will commit and push changes to the docs if a PR only had changes in mermaid/src/docs.
13. We have been merging some PRs to develop and letting the action take care of building it and updating /docs.

---

What now?

Facts:

• The users have an excellent documentation site at mermaid.js.org.
• We've had requests and "comments" from maintainers and contributors to remove the generated docs.
• The docs requirement is adding extra friction to simple PRs.

I think keeping the generated docs in the repo is doing more harm than good (to the maintainers and contributors, not to the users).

As none of our active maintainers have expressed any concerns against this until now, we should be good to go. Will check with @knsv too.

But, we should still use proper Markdown to write the docs, instead of any framework flavoured ones. (E.g.: the note, warning labels in vitepress). This will ensure that we can switch to another framework when required.

[emersonbottero]

regarding the last part "But, we should still use proper Markdown to write the docs, instead of any framework flavored ones. (E.g.: the note, warning labels in vitepress). This will ensure that we can switch to another framework when required." we can always write transformers to make an github specify markdown to fit the docs library... 🤓

But.. saying that , there is several vitepress specific features I would use.. I think Code Groups would improve the a lot the really long pages as we could show a compact format.

[nirname]

@emersonbottero code groups is a nice feature, recently I implemented similar behavior using pure Vue component, that was a bunch of switch buttons, that allowed to update your page dynamically and display only those things that are related to a current selected mode. So the similar idea was applied to the whole document, not to the code groups only. But sticking to the pure markdown is what allows us to switch seamlessly between different frameworks. After discussion we decided to remove it, because we cannot embed all those features in our core documentation, unfortunately. I personally think that our needs have exceeded markdown facilities. But we cannot utilize framework-specific features to meed our demands. The way to go is to add custom extensions that are compatible with markdown (pure js) or replace markdown with something equally widespread and simple.

[emersonbottero]

I would use more types of alert

and.. imagine that I could transform sections in tabs..

we could define a rule like : in the summary is the divisor for tabs

then something like

Group1: config.js

```js /** * @type {import('vitepress').UserConfig} */ const config = { // ... } export default config ```

Group1: config.ts

```ts [config.ts] import type { UserConfig } from 'vitepress' const config: UserConfig = { // ... } export default config ```

would use the default section in github but it will look like in the docs.. just trying to give options here with what we can use, if it does not fit the mermaid team I wont be sad, you guys are already using my docs.. 😎

[nirname]

Now I am revising "contributing to documentation" section and it says "we allow documentation PR's to target master branch, if it is for documentation only". This means, that our docs have different release cycle from mermaid package. Any thoughts?

[aloisklink]

"we allow documentation PR's to target master branch, if it is for documentation only". This means, that our docs have different release cycle from mermaid package.

I think that's the point of it. Improved documentation doesn't need to wait for the next Mermaid release, we can just release it straight away by putting it on master.

Although, I'm not a big fan of targeting the master branch for big PRs, since it makes it easy to create merge conflicts then. For smaller PRs, though, I guess it's okay.

I'll much rather have something like what Docusaurus-based websites often have: a next version of the docs that contains the unreleased doc changes: https://pptr.dev/next/

[nirname]

@aloisklink My suggestion is to make guidelines in the way that targeting development is the default regardless of type of change. Maybe even a hot fix is not an exception either. By doing that we are

• keeping guidelines simple
• avoid messing with PRs that target master branch
• avoid some unnecessary master to develop merges

upd: My bad, that discussion should've been there #2910