Incompatibility with Prism.js

[karlhorky]

Hey @wooorm ! 👋 Long time no speak! Hope you are well.

Input:

export default function Home() {
  return (
    <div>
      <FilterInput
        options={genreList} // genreList from assets directory
        value={genreFilter} // genreFilter from useState()
        filterSetter={setGenreFilter} // setGenreFilter from useState()
        name="genre"
      />
    </div>
  );
}

Output (refractor):

Demo (try the Run button at the top): https://replit.com/@karlhorky/FrequentWickedDemos#index.mjs

{
  "type": "root",
  "children": [
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "export"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "default"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "function"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "function"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "Home"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "("
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": ")"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "\n  "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "return"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "("
        }
      ]
    },
    {
      "type": "text",
      "value": "\n    "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "tag"
        ]
      },
      "children": [
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "tag"
            ]
          },
          "children": [
            {
              "type": "element",
              "tagName": "span",
              "properties": {
                "className": [
                  "token",
                  "punctuation"
                ]
              },
              "children": [
                {
                  "type": "text",
                  "value": "<"
                }
              ]
            },
            {
              "type": "text",
              "value": "div"
            }
          ]
        },
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "punctuation"
            ]
          },
          "children": [
            {
              "type": "text",
              "value": ">"
            }
          ]
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "\n      <FilterInput\n        options="
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "genreList"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": " // genreList from assets directory\n        value="
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "genreFilter"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": " // genreFilter from useState()\n        filterSetter="
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "setGenreFilter"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": " // setGenreFilter from useState()\n        name=\"genre\"\n      />\n    "
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "tag"
        ]
      },
      "children": [
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "tag"
            ]
          },
          "children": [
            {
              "type": "element",
              "tagName": "span",
              "properties": {
                "className": [
                  "token",
                  "punctuation"
                ]
              },
              "children": [
                {
                  "type": "text",
                  "value": "</"
                }
              ]
            },
            {
              "type": "text",
              "value": "div"
            }
          ]
        },
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "punctuation"
            ]
          },
          "children": [
            {
              "type": "text",
              "value": ">"
            }
          ]
        }
      ]
    },
    {
      "type": "text",
      "value": "\n  "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": ")"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": ";"
        }
      ]
    },
    {
      "type": "text",
      "value": "\n"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    }
  ]
}

I'm assuming this leads to this output, which I can observe with react-refractor:

---

Testing on https://prismjs.com/test.html#language=jsx, it looks like:

[wooorm]

Hi there! All is well. Busy with going all in on ESM — halfway there, about to publish all micromark things.

I dove in a little bit: It seems that FilterInput is in this case never seen as a tag, and I’m guessing it should hit this branch?

Can you share the Prism HTML you get?

Here’s what I get from refractor, as HTML:

<span class="token keyword module">export</span> <span class="token keyword module">default</span> <span class="token keyword">function</span> <span class="token function"><span class="token maybe-class-name">Home</span></span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
  <span class="token keyword control-flow">return</span> <span class="token punctuation">(</span>
    <span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;</span>div</span><span class="token punctuation">></span></span><span class="token plain-text">
      &#x3C;FilterInput
        options=</span><span class="token punctuation">{</span>genreList<span class="token punctuation">}</span><span class="token plain-text"> // genreList from assets directory
        value=</span><span class="token punctuation">{</span>genreFilter<span class="token punctuation">}</span><span class="token plain-text"> // genreFilter from useState()
        filterSetter=</span><span class="token punctuation">{</span>setGenreFilter<span class="token punctuation">}</span><span class="token plain-text"> // setGenreFilter from useState()
        name="genre"
      />
    </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;/</span>div</span><span class="token punctuation">></span></span>
  <span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

[karlhorky]

Busy with going all in on ESM — halfway there, about to publish all micromark things.

Wow, respect, that's a big effort! Really appreciate it! 🙌

Can you share the Prism HTML you get?

Sure! Prism.js HTML from the Test Drive is here:

<span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">function</span> <span class="token function">Home</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
  <span class="token keyword">return</span> <span class="token punctuation">(</span>
    <span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>div</span><span class="token punctuation">&gt;</span></span><span class="token plain-text">
      </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span><span class="token class-name">FilterInput</span></span>
        <span class="token attr-name">options</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>genreList<span class="token punctuation">}</span></span> <span class="token comment">// genreList from assets directory</span>
        <span class="token attr-name">value</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>genreFilter<span class="token punctuation">}</span></span> <span class="token comment">// genreFilter from useState()</span>
        <span class="token attr-name">filterSetter</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span>setGenreFilter<span class="token punctuation">}</span></span> <span class="token comment">// setGenreFilter from useState()</span>
        <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>genre<span class="token punctuation">"</span></span>
      <span class="token punctuation">/&gt;</span></span><span class="token plain-text">
    </span><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>div</span><span class="token punctuation">&gt;</span></span>
  <span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

[wooorm]

Weird... I don’t understand it 🤷‍♂️

Help would be appreciated if anyone wants to dig in!

[karlhorky]

@RunDevelopment you wouldn't have a clue here, would you? I guess you're familiar with Prism, but do you have an idea why Refractor wouldn't be generating the same output?

[RunDevelopment]

you wouldn't have a clue here, would you?

Actually, I do. The fix for bug PrismJS/prism#2727 hasn't been released yet.

The original plan was that the next version - v1.24.0 - will be the last v1.x release of Prism. The Prism team decided this internally in early 2021. The plan was to figure out what v2.0 should look like, release v1.24.0, and then start working on v2.0. However, we simply hadn't had the time yet.

However, I also see the need for a new release. The last release was 6 months ago. I will propose to finish a new release by the end of the week.

[karlhorky]

Ah ok, and the Test Drive uses the master branch on GitHub (showing the unreleased fix from the PR)?

[RunDevelopment]

Test Drive uses the master branch on GitHub (showing the unreleased fix from the PR)?

Yes, exactly. Everything on the website comes directly from master.

[wooorm]

Glad to know where the root cause is, and that it’s already fixed!

I’ll close this for now as it’s already fixed upstream and I’ll pull it in when the next Prism release happens!

Thanks @RunDevelopment and @karlhorky!