Docusaurs: Script the old syntaxes to the new form

[Jwaegebaert]

We are migrating our documentation from MKDocs to Docusaurus, but the syntax used by MKDocs differs from that used by Docusaurus, making migration challenging. To automate the formatting process, we need a script that can identify and modify all relevant syntax in our documentation files. This issue involves creating and testing the script on our existing documentation to ensure it works correctly and saves time and effort.

Automated tasks

• [x] Resolve the admonition syntaxes.
• [x] Resolve the command output syntaxes. Take a look at planner bucket add to see how this has been achieved. https://github.com/duanwilliam/docusaurus-remark-plugin-codetabs
• [x] Resolve the embeddings used for global options to the new syntax.
  • [ ] Use a general syntax to allow us to import from different files
• [x] Resolve the option descriptions for all the commands.
• [x] Take front matter into account

Manual to-do list:

• [x] Resolve options with no unique options. Search on the following:

## Options

- [x] Tabs where quadruple backticks occur. e.q. 'project-doctor'
- [x] Options that don't contain global. e.q. 'project-doctor'
- [x] (Bug in script) With the upgrade to mdx. Some internal links extension names end with more `x` chars. Search on `mdxx`
- [ ] ~~Delete the file `_global.md` and create under the directory `src/docs` the file `globalOptions.md`. Resources below.~~
- [x] Update `_global.mdx` with definition-list code-block
- [x] Add a new file to the directory `src/remark` named `definitionLists.mjs`. Resources below.
- [x] Install the required packages for the remark plugin `npm i unist-util-visit@2.0.3 micromark`. Note: validate if dev-dependency is enough
- [x] Update `docusaurus.config.js` to include remark plugin. Resources below.

### Resources

#### definitionLists.mjs

``` mjs
import visit from 'unist-util-visit'; // "unist-util-visit": "^2.0.3"
import { micromark } from 'micromark'; // "micromark": "^3.1.0"

/**
 * Turns a "```md defintion-list" code block into a definition list
 */

export default function plugin() {
  const transformer = (root) => {
    visit(root, 'code', (node, index, parent) => {
      if (!node.meta?.includes('defintion-list')) {
        return;
      }

      const {value} = node;
      const listItems = value.split('\n').filter(x => x);
      const items = listItems.map((listItem, i) => {
        const content = micromark(i % 2 ? listItem.substring(2, listItem.length) : listItem);

        if (i % 2) {
          return `<dd>${content}</dd>`;
        } 
        else {
          return `<dt>${content}</dt>`;
        }
      });

      parent.children[index] = {
        type: 'html',
        value: `<dl>${items.join('')}</dl>`
      };
    });
  };

  return transformer;
}

docusaurus.config.js

// ...

const config = {
  // ...
  presets: [
    [
      'classic',
      /** @type {import('@docusaurus/preset-classic').Options} */
      ({
        docs: {
          routeBasePath: '/',
          sidebarPath: require.resolve('./sidebars.js'),
          editUrl: 'https://github.com/pnp/cli-microsoft365/blob/main/docs',
          showLastUpdateTime: true,
          remarkPlugins: []
        },
        blog: false,
        theme: {
          customCss: require.resolve('./src/css/custom.css'),
        }
      })
    ]
  ],
  // ...
};

async function createConfig() {
  const definitionLists = (await import('./src/remark/definitionLists.mjs')).default;
  config.presets[0][1].docs.remarkPlugins.push(definitionLists);
  return config;
}

module.exports = createConfig;

---

Related to #4396

[Jwaegebaert]

Our option descriptions are currently in description list format in markdown, which are then processed into dt/dd tags. Unfortunately, Docusaurus does not yet support this feature as described in https://www.markdownguide.org/tools/docusaurus/#docusaurus-markdown-support. I attempted to resolve this by using the remark-definition-list plugin, but it is incompatible with the current version of Docusaurus since it relies on a different version of Remark. While Docusaurus v3 has plans to upgrade its Remarkable processor, it is unclear whether description lists will be supported in markdown. Given this situation, we must decide whether to wait for the new version or explore alternative options. @pnp/cli-for-microsoft-365-maintainers how do you think about this?

[milanholemans]

dt/dd tags would be awesome and is very clean imo. But I think this is not really a blocking issue. Maybe there are similar/better alternatives

[waldekmastykarz]

Is there a way for us to build a custom plugin or would that be too much work?

[Jwaegebaert]

Is there a way for us to build a custom plugin or would that be too much work?

Yeah, I've had the same thought about it. I'll do some digging and see what I can whip up 😄

[Jwaegebaert]

After spending some time thinking about it, I managed to make a plugin that can create definition lists. The way to use it is a bit different now - to define a definition list, you need to put it inside a code block that looks like this:

## Options

```md defintion-list
`-n, --name <name>`
: Name of the bucket to add.

`--planId [planId]`
: ID of the plan to which the bucket belongs. Specify either `planId` or `planTitle` but not both.

`--planTitle [planTitle]`
: Title of the plan to which the bucket belongs. Specify either `planId` or `planTitle` but not both.

`--ownerGroupId [ownerGroupId]`
: ID of the group to which the plan belongs. Specify `ownerGroupId` or `ownerGroupName` when using `planTitle`.

`--ownerGroupName [ownerGroupName]`
: Name of the group to which the plan belongs. Specify `ownerGroupId` or `ownerGroupName` when using `planTitle`.

`--orderHint [orderHint]`
: Hint used to order items of this type in a list view. The format is defined as outlined [here](https://docs.microsoft.com/en-us/graph/api/resources/planner-order-hint-format?view=graph-rest-1.0).

This in place will be processed by the custom plugin to our usual dt/dd tag format 😄 

![afbeelding](https://user-images.githubusercontent.com/38426621/224546110-1da17292-e250-44bd-b8d5-28d5986cb01b.png)

The new syntax may be a bit bothersome, but it'll be worth it once Docusaurus updates its remark processor. Then, we can simply get rid of these codeblocks and go back to using our old syntaxes again.

[milanholemans]

Awesome job @Jwaegebaert, looking great!

[Jwaegebaert]

I have created a temporary pull request so that you can preview the expected outcome when the script is executed. This will allow you to review the syntax and suggest adjustments as necessary. 😄

[waldekmastykarz]

Won't this approach lead to trouble when we have to distinguish between actual code blocks and options?

[Jwaegebaert]

This shouldn't cause any problem as the plugin only formats code blocks that start with md defintion-list. All other code blocks are unaffected.

[waldekmastykarz]

We'll also need to add that logic to where we convert md to plain-text for display in the terminal, right?

[Jwaegebaert]

Yeah, that's correct. We need to implement an additional check when a code block starts with md defintion-list that we have to render them differently

[Jwaegebaert]

I'll create a new issue shortly where we will update --help print to accommodate the different syntaxes

[waldekmastykarz]

Delete the file _global.md and create under the directory src/docs the file globalOptions.md. Resources below.

If replace the md file with an html file, it will significantly complicate our terminal rendering. Couldn't we use an .mdx file with ```md definition-list which we can remove more easily and keep the current terminal rendering logic?

Install the required packages for the remark plugin npm i unist-util-visit@2.0.3 micromark

Shouldn't this be a dev dependency or do we need it for CLI runtime (eg. rendering help in terminal)?

[Jwaegebaert]

If replace the md file with an html file, it will significantly complicate our terminal rendering. Couldn't we use an .mdx file with ```md definition-list which we can remove more easily and keep the current terminal rendering logic?

There are some pros and cons to this different approach I used here. Because the markdown file is placed under the src folder, the remark plugin doesn't touch it. That is why I manually converted it to the dd/dt tags. I placed it under the src folder so I could use the alias @site/src when importing this file. import Global from '@site/src/docs/globalOptions.md'; This approach made it a lot easier to write the migration script.

As we don't have to touch the global options file a lot, this seemed quite fine to me but I didn't take into account the terminal rendering aspect.

Shouldn't this be a dev dependency or do we need it for CLI runtime (eg. rendering help in terminal)?

That's a good question. I've added it to the to-do list to check out if this would be good enough.

[waldekmastykarz]

Converting html to plain-text for use in terminal will be way more complicated than the currently used md format. I suggest that we either use the existing .md format, or if that's too hard, keep two versions: HTML and MD, which isn't ideal, but again easier to deal with that html > plain-text.