facebook / docusaurus

Easy to maintain open source documentation websites.
https://docusaurus.io
MIT License
56.63k stars 8.51k forks source link

Feature: referencing variables from markdown pages (placeholder substitution) #395

Closed julen closed 5 years ago

julen commented 6 years ago

First of all, thanks for sharing and supporting Docusaurus! It's been pretty easy to get started.

I'm posting a feature request — I haven't found anything similar in the issue tracker so forgive me if it's been discussed before.

Motivation: sometimes docs need to reference information such as version numbers, URLs etc. and having them manually typed in the markdown pages is not ideal because this type of information often gets referenced multiple times.

Taking that into account, it would be great if Docusaurus supported passing variables into pages (from e.g. siteConfig). It already supports special Markdown markup via the <AUTOGENERATED_TABLE_OF_CONTENTS> marker, and it could potentially do something similar for variables.

For example, typing in<VAR:projectName> would get replaced with the value of siteConfig.projectName. Ideally this works in code blocks too (```).

JoelMarcey commented 6 years ago

Interesting idea. We did something similar for nuclide.io using JS and referencing the package.json file. I think this is something that could definitely be implemented for some upcoming version of Docusaurus.

tibdex commented 6 years ago

That would be an awesome feature!

Julen mentioned "version numbers, URLs, etc." but it would also be great to insert automatically computed markup from JSDoc for instance. Thus, if you decide to implement it, it would be nice to support the injection of Markdown (or HTML) markup and not only scalar values.

gedeagas commented 6 years ago

HI, @JoelMarcey I am planning to take this after finishing https://github.com/facebook/Docusaurus/pull/694 but I think I will work with scalar values first. Is that okay?

JoelMarcey commented 6 years ago

@gedeagas Sounds great! I think this is a feature that can be done in increments.

zenflow commented 6 years ago

Isn't this another thing best left to markdown plugins?

endiliey commented 6 years ago

While I agree with @zenflow, if @gedeagas intends to build his own plugin, that's welcome too☺

gedeagas commented 6 years ago

Hi @zenflow @endiliey, I am planning to do just that 😄 . Making a plugin for remarkable to support this on docusaurus 📦

zenflow commented 6 years ago

Ok, but just to be clear: that plugin will not be included as a dependency of Docusaurus, right? It would be configured by users in their siteConfig.markdownPlugins?

I'm just a bit confused by the fact that this is a feature request for Docusaurus when nothing actually needs to change in Docusaurus for this to be supported

tibdex commented 6 years ago

Here's how variable injection can be done:

Put this in siteConfig.js:

const {Plugin: Embed} = require('remarkable-embed');

// Our custom remarkable plugin factory.
const createVariableInjectionPlugin = variables => {
  // `let` binding used to initialize the `Embed` plugin only once for efficiency.
  // See `if` statement below.
  let initializedPlugin;

  const embed = new Embed();
  embed.register({
    // Call the render method to process the corresponding variable with
    // the passed Remarkable instance.
    // -> the Markdown markup in the variable will be converted to HTML.
    inject: key => initializedPlugin.render(variables[key])
  });

  return (md, options) => {
    if (!initializedPlugin) {
      initializedPlugin = {
        render: md.render.bind(md),
        hook: embed.hook(md, options)
      };
    }

    return initializedPlugin.hook;
  };
};

const siteVariables = {
  scalar: 'https://example.com',
  // Since the variables are processed by Docusaurus's Markdown converter,
  // this will become a nice syntax-highlighted code block.
  markdown: [
    '```javascript',
    'const highlighted = true;',
    '```',
  ].join('\n'),
  // We can use HTML directly too as HTML is valid Markdown.
  html: [
    '<details>',
    '  <summary>More details</summary>',
    '  <pre>Some details</pre>',
    '</details>'
  ].join('\n')
};

const siteConfig = {
  markdownPlugins: [
    createVariableInjectionPlugin(siteVariables)
  ],

  // rest of the config
};

module.exports = siteConfig;

Then, in one of the doc page, write:

## Scalar injection
{@inject: scalar}

## Markdown injection
{@inject: markdown}

## HTML injection
{@inject: html}

And this is what Docusaurus would render:

screenshot-2018-5-30 variable injection demo test site

As you can see, by leveraging remarkable-embed, the implementation of the custom plugin for variable injection is quite small. I'm not even sure it's worth making a dedicated npm package out of it.

The only drawback I can see is that the injected markup isn't pre-processed by Docusaurus. For instance, if one of the variables contains a Markdown title such as ## Custom title, this title won't appear in the secondary on-page navigation.

gedeagas commented 6 years ago

Awesome @tibdex. I guess my comment above is obsolete 😄

cmdcolin commented 6 years ago

This is really sweet tibdex...it is not replacing variables that are inside of a markdown code block though. I guess those are kind of regarded as literals but as I might want some code to depend on a variable, is there anything I should do?

Edit: also realized the original post requests having this feature available in code blocks

endiliey commented 5 years ago

This issue is closed due to stale activity.

If you think there is a need for it, please open up a new issue with new template.

camcamd commented 5 years ago

First of all, thanks @tibdex for this plugin example ! Is there a way to have the injected markdown processed by docusaurus, so that we could inject markdown with docusaurus code tabs for instance ?

tibdex commented 5 years ago

An option is to preprocess the markdown files outside of Docusaurus. You can develop a pipeline that reads the unprocessed markdown files from another directory, process them, write them to the Docusaurus's markdown files directory, and then build your website with Docusaurus.

anidotnet commented 4 years ago

Is there any official way to reference variables in v2?

JoelMarcey commented 4 years ago

What type of variables are you trying to reference?

anidotnet commented 4 years ago

Hi @JoelMarcey, I want to declare a global javascript variable, say version, which I can reference and replace in doc files. This will be useful while writing download link or library version in the doc. I don't have to manually change the version in the doc every time I make a release. So is there any support of such kind in v2?

JoelMarcey commented 4 years ago

@yangshun Did this ever get implemented for v2, do you know?

cc @slorber

yangshun commented 4 years ago

Yeah you can write JS in MDX so you can import anything and use variables.

JoelMarcey commented 4 years ago

https://v2.docusaurus.io/docs/markdown-features

@anidotnet See if you can get that to work, and, if you are so inclined, you could send a pull request to update the documentation to talk about how to make it work.

anidotnet commented 4 years ago

Hi @JoelMarcey thanks. But I never used react and a newbie in js world. So if I write my version in a .env file like

VERSION=1.0

or write it in a version.js file at the root of the project like

export const siteVariables = {
    versionNumber: '1.0.0'
};

how do I use any one of these in an mdx file?

Mainly I am trying to replace a javascript variable in a code block like

<Tabs
  groupId="installation"
  defaultValue="java"
  values={[
    { label: 'Java', value: 'java', },
  ]
}>
<TabItem value="java">
    ```groovy
compile 'mylib:mylib:$versionNumber'



Any help would be highly appreciated.
yangshun commented 4 years ago

I don't know if you can use variables within the markdown syntax (fenced code blocks), but you can use it within JSX.

import {siteVariables} from '@site/src/versions';

// ...

<Tabs
  groupId="installation"
  defaultValue="java"
  values={[
    { label: 'Java', value: 'java', },
  ]
}>
<TabItem value="java">
// Cannot use Markdown syntax here
<code>compile 'mylib:mylib:{siteVariables.versionNumber}'</code>

</TabItem>
</Tabs>
anidotnet commented 4 years ago

@yangshun thanks a lot for the pointer. I have used below code to achieve seamless effect of fenced code block.

<pre><code parentName="pre" {...{
            "className": "language-groovy"
          }}>{`compile 'mylib:mylib:${siteVariables.versionNumber}'
`}</code></pre>

Thanks again for prompt help.

kaytwo commented 3 years ago

For anyone looking for a succinct alternative to these options, as of at least Docusaurus 2.0.0-alpha.72, you can do this:

---
title: Page title
restofsentence: variable substitution from the front matter works like this
---

export const morewords = "use variables exported on the page"

# hello

<>The point of this comment is that {frontMatter.restofsentence}.<br />I can also {morewords}</>.

Fully inline substitution or multiline doesn't work (but may work once Docusaurus bumps mdx to v2. For those of us coming from Jekyll, this approach works for very basic frontmatter substitution within the document.

ibraheemdev commented 3 years ago

Is it possible to reference the site config inside of a markdown code block?

slorber commented 3 years ago

Not sure what you mean @ibraheemdev, any example code snippet to share and the expected result you want?

ibraheemdev commented 3 years ago

@slorber

// foo.md

const foo = 121232;

'''lang
function main() {
   dostuff({foo}) // interpolate the outer variable `foo` in to the code block somehow
}

// renders
function main() {
   dostuff(121232)
}
'''
slorber commented 3 years ago

This is not possible with md syntax but you can use MDX, as any exported value becomes available for JSX code blocks

// foo.md

export const foo = 121232;

<code>foo = {foo}</code>
federico-terzi commented 3 years ago

If anyone wants to read a variable defined in docusaurus.config.js from a markdown file, you can define the file as MDX instead of MD

And then:

This is _markdown_

import siteConfig from '/docusaurus.config.js';

The website is called = {siteConfig.title}

Took me a while to figure out

slorber commented 3 years ago

Technically we should be able to add extra variables in the MDX scope, so that you can use them inside MDX, like the docusaurus context (which contains the site config) and any data Docusaurus has internally (like metadatas of the mdx file itself, path, version, last update date...). That's worth opening a dedicated issue if this is a common need.

derekm commented 2 years ago

For variables in code blocks, see also here: https://github.com/mdx-js/mdx/issues/1095#issuecomment-967511279

seladb commented 2 years ago

Is there a way to do that in links? For example

// foo.md

import {siteVariables} from '@site/src/versions';

[a link to the latest version](https://github.com/user/project/blob/{siteVariables.latestVersion}/file)

This renders the URL to https://github.com/user/project/blob/{siteVariables.latestVersion}/file

Josh-Cena commented 2 years ago

@seladb You need to use JSX:

import {siteVariables} from '@site/src/versions';

<a href={`https://github.com/user/project/blob/${siteVariables.latestVersion}/file`}>a link to the latest version</a>
seladb commented 2 years ago

thanks @Josh-Cena , is there a way to do it with markdown syntax or the only way is to replace this with HTML <a></a> tag?

Josh-Cena commented 2 years ago

is there a way to do it with markdown syntax or the only way is to replace this with HTML tag?

Yep, JSX is the only way. The equivalent MD syntax is valid on its own, you can think of that as

<a href="https://github.com/user/project/blob/{siteVariables.latestVersion}/file">a link to the latest version</a>
slorber commented 2 years ago

You can also build your own remark plugin to pre-process the file

boevski commented 2 years ago

We need an elegant variable insertion approach from Docusaurus - it's predominantly a documentation solution after all. For example, keeping product names in variables is a good practice for rebranding and consistency reasons. Inserting components and referencing variables using MDX is far from practical for a string that repeats potentially dozens of times in an article.

I'd suggest having something like this which is exported by default to all MD files:

  // docusaurus.conig.js, or variables.js, or similar
  productName: 'Docusaurus`,
  //...

Then, in the MD file we could simply:

// article.md
Understand {config.productName} in 5 minutes by playing!

Create a new {config.productName} site and follow the very short embedded tutorial.

Install [Node.js](https://nodejs.org/en/download/) and create a new {config.productName} site:

Or maybe:

Understand {{config.poductName}} in 5 minutes by playing!

Or even:

Understand config:::poductName in 5 minutes by playing!
slorber commented 2 years ago

@boevski as said you can already build your own remark plugin for this

I understand this is a quite often requested feature and we plan to create one to make it easier for all, but in the meantime you shouldn't be blocked and could implement this feature in userland

Zenahr commented 1 year ago

@slorber Excuse the revive of this issue. Is this a feature that's still on the Docusaurus roadmap? Supporting reusable blocks of info would bring Docusaurus very close to the reusable content paradigm prevalent among most modern documentation systems. IMO it does make a lot of sense to provide 1st citizen to this feature and I'll happily aid in speccing this out if that's wanted.

I think @boevski's proposal gets very close to an optimal, easy to use proposal. (https://github.com/facebook/docusaurus/issues/395#issuecomment-1220761325)

If this isn't something the Docusaurus core team wants to prioritize on the roadmap I'd appreciate some communication over that. When we know it's not prioritized we as a community could work on a plugin to support this feature.

slorber commented 1 year ago

Is this a feature that's still on the Docusaurus roadmap?

Yes

Supporting reusable blocks of info would bring Docusaurus very close to the reusable content paradigm prevalent among most modern documentation systems.

There's a difference between reusable blocks and simple variable substitution. Both can be achieved with a custom remark plugin.

IMO it does make a lot of sense to provide 1st citizen to this feature and I'll happily aid in speccing this out if that's wanted.

If this isn't something the Docusaurus core team wants to prioritize on the roadmap I'd appreciate some communication over that. When we know it's not prioritized we as a community could work on a plugin to support this feature.

It is on the roadmap, as many other things. It doesn't mean it will be worked on soon, considering it's possible to build this in userland already thanks to a remark plugin.

The best thing you can do to help is... to actually build the remark plugin and share your results with us.

If it works fine for you, we can eventually add it as a core feature.

There's nothing we can do that you can't. Our core Docusaurus solution would also be... a remark plugin 😅

Again, I encourage anyone willing to solve this to create a custom remark plugin 😄


Note that there are already some existing remark plugins you can take inspiration from, and that you can find on Google or npm search results.

For example this Gatsby plugin mentioned here: https://github.com/facebook/docusaurus/issues/5700#issuecomment-1351790489

CleanShot 2022-12-15 at 13 00 59@2x


I think @boevski's proposal gets very close to an optimal, easy to use proposal. (https://github.com/facebook/docusaurus/issues/395#issuecomment-1220761325)

This proposal is using {} syntax, which is something MDX 2 uses for reading JSX expressions. It is probably not the ideal solution/syntax because it will conflict with regular MDX features (eventually IDE support if they implement a good MDX plugin)

If we add support for things like {context.siteConfig.name}, (I'm thinking of doing that actually), this will be a first-class MDX feature, and not really a simple string variable substitution system.

For example, you could do more powerful things such as:

# Title

{context.siteConfig.name.toUpperCase() + " is the best site - " + context.siteConfig.customFields.someCustomText}

For these reasons it becomes important to distinguish between the different use-cases that are discussed in this issue:

All 3 things are likely to require different solutions here

Zenahr commented 1 year ago

@slorber Time to get my hands dirty then. Thanks for your pointers. Great reference.

Zenahr commented 1 year ago

@boevski this might be of interest to you.

@slorber I've finished a first working draft for naïve variable replacement. I've opted for ^ as a prefix for now. It probably makes sense to expose the prefix as a plugin option for users, while recommending sensible options. Let me know your thoughts on this. Outlined below is every significant part of the solution, which, to be far, is fairly naïve at the moment.

To do a naive check on the scalability, I've spun up a docusaurus instance with a replacement map of 100K entries. There was no noticeable performance impact on the serve time.

plugin (src/remark/variable-injector.js):

const visit = require('unist-util-visit');

const plugin = (options) => {
  const transformer = async (ast) => {
    visit(ast, 'text', (node) => {
      // Replace all occurrences of ^varName with the value of varName
      node.value = node.value.replace(/\^([A-Z_]+)/g, (match, varName) => {
        return options.replacements[varName] || match;
      });
    });
  };
  return transformer;
};

module.exports = plugin;

docusaurus.config.js:


const variableInjector = require('./src/remark/variable-injector')

/** @type {import('@docusaurus/types').Config} */
const config = {
...
  presets: [
    [
      'classic',
      /** @type {import('@docusaurus/preset-classic').Options} */
      ({
        docs: {
            remarkPlugins: [
                        [variableInjector, {
                            replacements: {
                                COMPANY: 'ACME Inc.',
                                CURRENT_YEAR: new Date().getFullYear()
                            }
                        }]
                    ],

Usage (inside any markdown or MDX file):

^COMPANY

^CURRENT_YEAR

# ^COMPANY

This solves only the simple variable substitution part of the equation.

Let me know what you'd require for this to be integrated into Docusaurus as a core feature. I'll try and see to releasing this as a docusaurus plugin, although I'm not sure I'll get that done easily (I'm not a JS dev by any stretch of the imagination). At the very least I hope this attempt of mine can serve inspiration for someone more adapt to implement a more well-designed approach, e.g., with the option to pass a file reference (JSON, JS, TXT) to the plugin options and other neat things.

Addendum: For increased user comfort and tooling it makes sense to provide a VSCode plugin that automatically looks up and recommends variables set inside the replacements map. This way, substitutions stay manageable by writers. There's definitely other ways of giving variables structure. E.g., namespacing via :: (borrowing from C and C++ syntax):

fictitious usage: ^BRANDING::COMPANY_NAME ^HARDWARE::LATEST_MODEL vs. ^SOFTWARE::LATEST_MODEL

fictitious definition with namespaces:

const namespaced_replacements = {
  BRANDING: {
    COMPANY_NAME: 'ACME Inc.',
  },
  HARDWARE: {
    LATEST_MODEL: 'ACME-HW-1084'
  },
  SOFTWARE: {
    LATEST_MODEL: 'ACME-SW-3894'
  }
}
boevski commented 1 year ago

@Zenahr This is great work! The community thanks thee!

I like the idea to use upper case for variables. This could help dodge a couple of corner cases.

So tried it out (sorry for the delay) and everything works fine. I made these checks:

In other thoughts, I'm wondering if a syntax like VAR::COMPANY (regex: /(VAR::[A-Z_]+)/g) would make it even more safe. And it will have the benefit of searching for variables more easily (as VAR:: is unique enough, unlike ^).

What do you think?

slorber commented 1 year ago

Thanks

Let me know what you'd require for this to be integrated into Docusaurus as a core feature.

Adding this to Docusaurus doesn't mean the plugin code have to live in the Docusaurus codebase. Ideally this plugin should work with all the tools based on mdx/remark, and eventually we add this plugin to Docusaurus to make it more convenient, only as a "shortcut".

If a good Remark plugin exists, it's not a top priority to add it to Docusaurus anyway, because... it's not very complicated to register a remark plugin to Docusaurus.

Zenahr commented 1 year ago

@slorber Makes sense. Thanks!

Zenahr commented 1 year ago

@boevski Yeah, I like the C-inspired namespacing syntax! Thanks for figuring out that adding it to beforeDefaultPlugins solves issues I haven't even discovered before.

I hope I can find the time to make a dedicated repo for this feature and hopefully add an interface for handling translations.

irreal commented 1 year ago

@Zenahr @boevski Heads up, moving the plugin to "beforeDefaultRemarkPlugins" fixes replacement inside the TOC, but it does not fix the replacement inside an auto generated category doc page.

See attached image image

It works fine in TOC and regular content.

(I'm using VAR:: as the prefix, instead of ^)

Zenahr commented 1 year ago

@irreal thanks for the heads-up! Some native docusaurus components might be exempt of remark processing. I wonder if someone from the core team could chime in and let us know if this is the case and how we could potentially circumvent this.

@Josh-Cena Do you happen to have a quick fix/insight into what's happening here?

irreal commented 1 year ago

Interestingly, same issue is present when using jsx too.

I switched the contents to <p> {1+1} blah blah blah</p> just to test it out.

In generated category page: image

On the actual page: image

Josh-Cena commented 1 year ago

@irreal The extraction of title and description uses a pure regex-based approach without attempting to properly parse or evaluate MDX. This is only a convenience feature and is not meant to be perfectly functional. If you want to provide a more readable description, you should add a description front matter.