registerEditorComponent Markdown fields stop working on newline entry

[scwambach]

- Do you want to request a feature or report a bug? bug

- What is the current behavior? When rendering a registerEditorComponent, using the Markdown field, that has multiple lines the preview breaks.

- If the current behavior is a bug, please provide the steps to reproduce. Create an Editor Component that has a Markdown field and make sure the field has content with multiple lines.

Example:

CMS.registerEditorComponent({
  // Internal id of the component
  id: "about-section",
  // Visible label
  label: "About Section",
  // Fields the user need to fill out when adding an instance of the component
  fields: [{
    name: 'heading',
    label: 'Heading',
    widget: 'string'
  }, {
    name: 'image',
    label: 'Image',
    widget: 'image'
  }, {
    label: "Widget Content",
    name: "widget_content",
    widget: "markdown"
  }],
  // Pattern to identify a block as being an instance of this component
  pattern: /^{{% about-section heading="(.*)" image="(.*)" %}}((.|\n)*){{% \/about-section %}}$/,
  // Function to extract data elements from the regexp match
  fromBlock: function(match) {
    console.log(match[3]);
    return {
      heading: match[1],
      image: match[2],
      widget_content: match[3]
    };
  },
  // Function to create a text block from an instance of this component
  toBlock: function(obj) {
    return '{{% about-section heading="'+ obj.heading +'" image="'+ obj.image + '" %}}' + obj.widget_content + '{{% /about-section %}}';
  },
  // Preview output for this component. Can either be a string or a React component
  // (component gives better render performance)
  toPreview: function(obj) {
    return (
      '<div class="about-block clearfix"><div class="information"><h2>'+ obj.heading +'</h2>'+ obj.widget_content +'</div><div class="images"><img src="' + obj.image + '"></div></div>'
    );
  }
});

- What is the expected behavior? To render markdown correctly.

- Please mention your CMS, node.js, and operating system version. Netlify CMS version 1.0.4 Node.js 5.5.1 HUGO 0.30.2 macOS Sierra 10.12.6

- Please link or paste your config.yml below if applicable.

backend:
  name: github
  repo: scwambach/sprout # Path to your GitHub repository
  branch: master # Branch to update (master by default)

publish_mode: editorial_workflow

media_folder: "static/images/uploads"
public_folder: "/images/uploads"

collections:

  ##########################################
  ################## BLOG ##################
  ##########################################

  - name: "thoughts" # Used in routes, e.g., /admin/collections/thoughts
    label: "Thoughts" # Used in the UI
    folder: "/content/thoughts" # The path to the folder where the documents are stored
    create: true # Allow users to create new documents in this collection
    slug: "{{slug}}" # Filename template, e.g., YYYY-MM-DD-title.md
    fields: # The fields for each document, usually in front matter
      - {label: "Title", name: "title", widget: "string"}
      - {label: "Publish Date", name: "date", widget: "datetime"}
      - {label: "Draft", name: "draft", widget: "boolean", default: true}
      - {label: "Featured Image", required: false, name: "featured_image", widget: "image"}
      - label: "Extra Images"
        name: "extra_images"
        widget: "list"
        fields:
          - label: Image
            name: img
            widget: image
      - {label: "Body", name: "body", widget: "markdown"}

  ###########################################
  ################ PORTFOLIO ################
  ###########################################

  - name: "portfolio" # Used in routes, e.g., /admin/collections/portfolio
    label: "Portfolio" # Used in the UI
    folder: "/content/portfolio" # The path to the folder where the documents are stored
    create: true # Allow users to create new documents in this collection
    slug: "{{slug}}" # Filename template, e.g., YYYY-MM-DD-title.md
    fields: # The fields for each document, usually in front matter
      - {label: "Title", name: "title", widget: "string"}
      - {label: "Publish Date", name: "date", widget: "datetime"}
      - {label: "Draft", name: "draft", widget: "boolean", default: true}
      - {label: "Featured Image", name: "featured_image", widget: "image"}
      - {label: "Short Name", name: "short_name", widget: "string"}
      - label: "Order Number"
        name: "order_number"
        widget: "number"
        default: 0
        valueType: "int"
      - label: "Credits"
        name: "credits"
        widget: "list"
        fields:
          - {label: "Title", name: "title", widget: "string"}
          - {label: "Copy", name: "copy", widget: "string"}
          - {label: "Link", name: "link", widget: "string"}
      - label: "Gallery"
        name: "gallery"
        widget: "list"
        fields:
          - label: Image
            name: img
            widget: image
      - {label: "Body", name: "body", widget: "markdown"}

  #########################################
  ################# PAGES #################
  #########################################

  - name: "pages" # Used in routes, e.g., /admin/collections/pages
    label: "Pages" # Used in the UI
    folder: "/content" # The path to the folder where the documents are stored
    create: true # Allow users to create new documents in this collection
    slug: "{{slug}}" # Filename template, e.g., YYYY-MM-DD-title.md
    fields: # The fields for each document, usually in front matter
      - {label: "Title", name: "title", widget: "string"}
      - {label: "Publish Date", name: "date", widget: "datetime"}
      - {label: "Draft", name: "draft", widget: "boolean", default: true}
      - {label: "Featured Image", required: false, name: "featured_image", widget: "image"}
      - {label: "Body", name: "body", widget: "markdown"}

  ##########################################
  ################ SETTINGS ################
  ##########################################

  - label: "Site Settings"
    name: "settings"
    editor:
      preview: false
    files:
      - label: "Global Settings"
        name: "global"
        file: "data/global.yml"
        fields:
          - {label: Favicon, name: favicon, widget: image}
          - {label: Footer Copy, name: footer_copy, widget: string}
          - label: Contact Info
            name: contact_info
            widget: object
            fields:
              - {label: Company Name, name: company_name, widget: string}
              - {label: Name, name: name, widget: string}
              - {label: Title, name: title, widget: string}
              - {label: Location, name: location, widget: string}
              - {label: Street, name: street, widget: string}
              - {label: City State, name: city_state, widget: string}
          - label: "Socials"
            name: "socials"
            widget: "list"
            fields:
              - label: Icon
                name: icon
                widget: string
              - label: URL
                name: url
                widget: string

[erquhart]

This is happening because the markdown widget parses content before shortcodes, and shortcode patterns are checked against individual blocks. When the content of a shortcode spans multiple blocks, it can never match the pattern.

We should address this by parsing shortcodes before parsing the Markdown string into an AST. This carries performance concerns that we'll need to be mindful of.

This fix is also a prerequisite for related issue #1001.

[papandreou]

I've also run into this while trying to use a custom editor component to recognize "islands" of yaml, eg.:

Here is some text

::: MyComponent
foo: bar
quux: baz
:::

Some more text.

I've specified my component's pattern as /::: MyComponent([\s\S]*)/, and the above case works fine. However, if I have leading whitespace or an empty line in the yaml block, it breaks:

::: MyComponent
foo: bar
quux: baz
array:
  - an item
  - another item
:::

in the above case it gets handed this:

::: MyComponent
foo: bar
quux: baz
array:

Similar to #1001 it also breaks if I have something resembling markdown styling inside the yaml.

I think I can work around this by bastardizing the yaml inside the component, but it would be immensely powerful if the components could have the ability to match the undigested markdown. Then I'd be able to set pattern to something like /^::: MyComponent\n([\s\S]+?\n)?:::\n/m.

I realize that there might be challenges associated with this as you'd probably have to hook in at multiple stages to hydrate all the components.

I also tried to use the ``` delimiter for my custom component, but it looks like that's already special cased, so I wasn't able to match that at all.

[erquhart]

So I remembered why editor components can't access raw markdown - it's for performance reasons. That's why we have a Remark plugin that transforms images back into markdown strings for parsing by the image editor component. Applying multiple RegExp patterns to a large markdown string can be a showstopper. By parsing the markdown before applying shortcode patterns, we:

1. Restrict pattern checking to top level nodes (shortcode patterns can't be nested currently).
2. Can begin patterns with ^, which is highly performant, and still have it apply to all potential shortcode nodes.

Ideally inline shortcodes would work and raw markdown would be accessible for editor components. Just need to innovate through that performance hurdle. Suggestions and PR's more than welcome here.

[papandreou]

@erquhart, for now I've resorted to forking netlify-cms and regexping the verbose format to something shortcode-like with a base64 blob: https://github.com/netlify/netlify-cms/compare/master...papandreou:netlify-cms-papandreou

And then have my editor widget recognize and serialize to that format:

      pattern: /^::: MyComponent (\S*)/,
      // Function to extract data elements from the regexp match
      fromBlock(match) {
        return jsyaml.load(base64Decode(match[1]));
      },
      // Function to create a text block from an instance of this component
      toBlock(obj) {
        return '::: MyComponent ' + base64Encode(jsyaml.safeDump(obj));
      },

It's a bit naive and annoying (and doesn't avoid the double regexping), but it seems to work.

[erquhart]

Gotcha, glad you were able to at least work out a stop gap.

[bjoernbg]

Is there any work planned for this issue. This is something that prevents us from trying out Netlify CMS for a serious website – we need components with multi line markdown content!

I don't really see the big performance issue either, since it would still be possible to take advantage of matching with /^…$/ – it's just that the string that's matched against is incomplete, so we would need to combine multiple lines of (processed) markdown into one string so our regex can match.

[erquhart]

@bjoernbg agreed - it's certainly possible, just takes thought and effort. Definitely open to contributions for this.

[eur2]

Anyone has an idea how to create an inline component based on selection (like Bold, Italic, Link, h1, etc.) with registerEditorComponent? It seems it's only made to add a new block Component into the markdown, no? I would like to create a custom modal component from a text selection (working very similar as a link). I didn't find any doc about selecting inline text with registerEditorComponent. Is it possible? Thanks!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[erquhart]

We're now allowing shortcode patterns to match against raw markdown, although I believe multiline is still disabled. We're closer, though. #2828 should help as well.

@eur2 inline isn't on the docket atm. Eventually, but not yet. All block level for now.

[papandreou]

We're now allowing shortcode patterns to match against raw markdown

@erquhart, that's amazing! Has that capability already been shipped?

[apolopena]

@erquhart Yes, is this in the production code now?

[davideforestali]

found a solution to that. I create my editor component in React and stringify it. the stringified component also has to match the 'pattern' regex. here's how:

import React from 'react';
import { renderToString } from 'react-dom/server';
import CMS from 'netlify-cms-app';

const TravelQuote = (props) => {
  const authorIsAnonymous = authorName && authorName.toLowerCase() === 'anonymous';
  const authorImg = props.authorPic ? props.authorPic : ""; // src needs to exist *
  // * the React components automatically removes the src attr if 
  // its attribute is undefined when it creates it.
  // With no src though, the component structur won't match the regex.

  return (
      `<div class="travel-quote">
        <p class="travel-quote__copy">{props.quote}</p>
        <span class="travel-quote__author">{props.authorName}</span>
        <img class="travel-quote__author-pic" src={authorImg} alt={props.authorName} />
      </div>`
  )
}

// create string from component to create the regex 
// for CMS.registerEditorComponent 'pattern'
const patternString = renderToString(
  `
<TravelQuote 
    quote='(.*)'
    authorName='(.*)' 
    authorPic='(.*)'/>
`
);

// create the regex
const travelQuoteRegex = new RegExp(patternString.replace(/\//g, '\\/'));

CMS.registerEditorComponent({
  id: "travelQuote",
  label: "Travel quote",
  fields: [
    { name: 'quote', label: 'Quote', widget: 'string' },
    { name: 'authorName', label: 'Author name', widget: 'string' },
    { name: 'authorPic', label: 'Author picture', widget: 'image' },
  ],
  pattern: travelQuoteRegex,
  fromBlock: function(match) {
    return {
      quote: match[1],
      authorName: match[2],
      authorPic: match[3],
    };
  },
  toBlock: function(obj) {
    return renderToString(
      `<TravelQuote 
        quote={obj.quote} 
        authorName={obj.authorName} 
        authorPic={obj.authorPic} />`
    )
  },
  toPreview: function(obj) {
    return renderToString(
      `<TravelQuote 
        quote={obj.quote} 
        authorName={obj.authorName} 
        authorPic={obj.authorPic} />`
    )
  },
});

// Note: renderToString() works only if the project is runned with the Production
// environment, with gatsby build and gatsby serve.

[erezrokah]

Thanks for sharing your solution @davideforestali!

[idrisadetunmbi]

Anyone has an idea how to create an inline component based on selection (like Bold, Italic, Link, h1, etc.) with registerEditorComponent? It seems it's only made to add a new block Component into the markdown, no? I would like to create a custom modal component from a text selection (working very similar as a link). I didn't find any doc about selecting inline text with registerEditorComponent. Is it possible? Thanks!

Any ideas if this is possible yet or is still in the works?

[martinjagodic]

Hi everyone

I found a regex pattern to match multiline markdown, maybe it helps someone.

From /^{{< text-image src="(.*)" alt="(.*)" >}}(.*){{< \/text-image >}}/

I changed it to /^{{< text-image src="(.*)" alt="(.*)" >}}(.*){{< \/text-image >}}/s.

The catch is in the last "s".

The entire component:

CMS.registerEditorComponent({
  id: 'text-image',
  label: 'Text & image',

  fields: [
    { name: 'src', label: 'Image', widget: 'image' },
    { name: 'alt', label: 'Image alt', widget: 'string', default: '', required: false },
    { name: 'inner', label: 'Content', widget: 'markdown' },
  ],

  pattern: /^{{< text-image src="(.*)" alt="(.*)" >}}(.*){{< \/text-image >}}/s,

  fromBlock (match) {
    return {
      src: match[1],
      alt: match[2],
      inner: match[3],
    }
  },

  toBlock (obj) {
    return `{{< text-image src="${obj.src}" alt="${obj.alt}" >}}${obj.inner}{{< /text-image >}}`
  },
})

[papandreou]

That's because the . character class doesn't match line terminators when the s flag isn't set: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Character_Classes#Types

In situations where you don't control the flags, you can use eg. [\s\S]* instead if .* to achieve the same thing (the "space or not space" hack).