[Attributed String] Preserve formatting tokens on render

[jsorge]

Please help prevent duplicate requests before submitting a new one:

• [x] I've searched other open/closed issues for duplicates before opening up this new issue.

Feature Request

If I have a markdown string such as:

This is some **really** great text I've got here!

When I render it as an attributed string using Down, the ** characters are removed and the styling is applied to the text inside the pair of ** tokens. I'd like to be able to preserve the tokens and display those in the attributed string, applying the styling to the tokens as well as the text they wrap.

Is your feature request related to a problem? Please describe.

I'm frustrated that I lose changes to the original markdown syntax that's rendered in my text view, so changes that I make to the text don't easily fit back to the document I'm modifying.

Describe the solution you'd like

I want to have the ability to use a text view (DownView or otherwise) and have it be editable, so that I can track the original source of my markdown text and apply changes using the standard text view delegate callbacks.

Describe alternatives you've considered

I attempted to pair Craig Hockenberry's MarkdownAttributedString library with Down to see if it could render back the attributed string but alas it could not on my initial try at it.

[iwasrobbed-ks]

Hi @jsorge 👋

If there is markdown in the string, it will render it as such. Have you tried using:

```markdown
**some text here**
```

Also, DownView renders within a web view, so you're likely better off using the newer attributed string API that gives you more control over everything: https://github.com/iwasrobbed/Down/blob/master/Source/Renderers/DownAttributedStringRenderable.swift#L48

You can likely use the same approach as the visitors for specific node types as well if you need more custom behavior. E.g. https://github.com/iwasrobbed/Down/blob/master/Source/AST/Visitors/AttributedStringVisitor.swift#L33

[jsorge]

Thanks for the response @rob-keepsafe 🙂

Yeah I am using DownTextView (I really should keep from filing issues late at night!). I have a custom styler in place for some of the style tweaks that I need to make and making use of the method you reference in the DownAttributedStringRenderable protocol. What I'm not understanding in the AttributedStringVisitor is that when I step through:

    public func visit(strong node: Strong) -> NSMutableAttributedString {
        let s = visitChildren(of: node).joined
        styler.style(strong: s)
        return s
    }

I see the value of s is a mutable attributed string already parsed, stripped of the ** or __ that made it strong. What part of the visitor chain do I need to customize to preserve those characters? (I'm also realizing that putting my original example in a code block may have also been confusing – I want to preserve all the token characters and not just those in code blocks. Sorry for the confusion!)

Thanks for the hep!

[iwasrobbed-ks]

Down uses cmark under the hood to transform any CommonMark Markdown string into an AST and then the attributed string visitor visits each node and styles it based on the node type.

So any string that goes in as Markdown will always be converted into a node and that's what will dictate/signal what was previously a ** in Markdown. Therefore, you can always use the visitor pattern (see Visitor.swift) to put back whichever Markdown text you want to preserve, or style it in some other custom way. An example of this is for how list bullets are generated via ListItemPrefixGenerator even though they're not part of the original text.

This is not something most people would want out-of-the-box, so feel free to override AttributedStringVisitor to add that behavior.

[iwasrobbed-ks]

Alternatively, I think you'd need to escape those characters and add additional information for the renderer to emphasize the text. Otherwise Markdown will just see that as text it needs to make bold

E.g.

**\*\*Hi there\*\***

Would give you **Hi there**

Whereas

**Hi there**

Will of course (just like any other Markdown renderer like GitHub) give you: Hi there

[johnxnguyen]

Hi @jsorge 👋 As @iwasrobbed-ks already mentioned, you could reinsert the tokens in your AttributedStringVisitor overrides. If you want the tokens to be styled the same as the content, just insert the tokens before calling the super implementation. This approach is simple and clean but it may not be what you're looking for.

I want to have the ability to use a text view (DownView or otherwise) and have it be editable, so that I can track the original source of my markdown text and apply changes using the standard text view delegate callbacks.

If you want to have an editor with markdown capability, then I'm afraid there isn't yet a good solution for this that involves Down. I don't think cmark was designed to preserve the original input string, since it appears some crucial information is lost during the parsing, such as how many line breaks there are between paragraphs. (I could be wrong, there may be some API in cmark that I'm not aware of). This means that there may always be some discrepancy between the input and output strings. This alone makes it not suitable for use in a user editable text view, because the user would expect only the styling to change and not the characters.

In short, if you only want to preserve some inline tokens, then subclassing the visitor and reinserting the tokens yourself is the best way to go. But if you need to preserve the characters of the input string after styling (for use in an editor), then I'm afraid that we don't have a solution for this.

[iwasrobbed-ks]

Going to close this one out as it diverges away from Down's primary use case and there are also a number of workarounds available for this edge case. Feel free to reopen or comment here if you have additional concerns