Marking code inside a codeblock will add the closing mark tag in the wrong position

dbadea-heits commented 5 months ago

Information

Plugin version: v1.14.0
Markdown-It version: v14.1.0
Environment: Chrome 119.0.6045.159

Details

Description

Attempting to mark code inside of a code block will misplace the mark closing tag, placing it before the next token starts, and if it is added after the last token ends it needs a whitespace to get added, other wise it is included in the last token. Usually the string-literal token is the easiest to reproduce this with.

Examples:

```ruby
<^><% %><^>
<% %>

<^>to: ""<^>

<^>const test = `hello<^>
world`;



In the last example we can see the mark closing tag is closed before the string literal starts:
<img width="343" alt="Screenshot 2024-06-03 at 12 35 53" src="https://github.com/digitalocean/do-markdownit/assets/147388674/3ac026e5-152b-4382-9563-8f9aaafb445d">

### Steps to reproduce
<!-- Replace this with exactly what you did to cause the bug. -->
Navigate to https://www.digitalocean.com/community/markdown
Clear the textbox
Paste one of the above examples inside of the textbox

### Expected behavior
<!-- Replace this with what you expected to happen. -->
Text inside of the <^> marking token gets marked in the resulting code block

GroophyLifefor commented 3 months ago

Hi @dbadea-heits ,

I was interested and did a little experiment, I added your case to the existing test cases and I was curious about the result because I was trying to catch the source of the problem.

New test case:

it("handles hightlight tags in code block with newline", () => {
    expect(md.render("```js\n<^>const test = `hello<^>\nworld`;\n```")).toBe(
        '<pre><code class="language-js"><mark>const test = `hello</mark>\nworld`;\n</code></pre>\n'
    );
});

as we see mark includes 'hello' part so code should be working, and you'd be surprised what you see test case passed 😄

also I looked the process for be sure and process too says It's okay. (everything is fine in the image below)

I couldn't find any proof because prism is a bit complicated, but I think the problem is that prism manipulates the code afterwards, it manipulates the code to make that string yellow when everything is fine.

As a result, I could not solve the problem, sorry, there seems to be a conflict for a different reason

MattIPv4 commented 3 months ago

@GroophyLifefor I don't think your test-case here is valid. Your expected result does not include any actual syntax highlighting from Prism.

GroophyLifefor commented 3 months ago

@MattIPv4 Hi Matt, Thank you for the information, I couldn't try it at the prism stage, so I was in between a lot, but when I couldn't find a problem in highlight, I thought it was a possibility.

MattIPv4 commented 3 months ago

This bug specifically relates to the interaction between Prism syntax highlighting, and the highlight markup 👍

GroophyLifefor commented 3 months ago

This bug specifically relates to the interaction between Prism syntax highlighting, and the highlight markup 👍

Yeah It's what I said in first comment, and if we talk about test-case, prism is an after-effect as far as I understand, so I couldn't test it or find out how to test it.

MattIPv4 commented 3 months ago

There are a whole set of dedicated test cases for it: https://github.com/digitalocean/do-markdownit/blob/master/modifiers/prismjs.test.js (specifically the HTML preservation block of tests)

digitalocean / do-markdownit