vsch / flexmark-java

CommonMark/Markdown Java parser with source level AST. CommonMark 0.28, emulation of: pegdown, kramdown, markdown.pl, MultiMarkdown. With HTML to MD, MD to PDF, MD to DOCX conversion modules.
BSD 2-Clause "Simplified" License
2.26k stars 267 forks source link

pegdown markdownToHtml and LinkRenderer class replacement in flexmark java #105

Closed rkanumola closed 5 years ago

rkanumola commented 7 years ago

Hi, I am looking for the replacement for the following logic in flexmark java, can you please suggest?

PegDownProcessor pegDownProcessor = new PegDownProcessor(Extensions.ALL - (headerLinks ? 0 : Extensions.ANCHORLINKS)

  • (hardwrap ? 0 : Extensions.HARDWRAPS) + (allowHtml ? 0 : Extensions.SUPPRESS_ALL_HTML)); try { processed = pegDownProcessor.markdownToHtml(data, new ConfluenceWikiLinkRenderer(info, xhtmlContent)); } catch (Exception exception) { exception.printStackTrace(); throw new MacroExecutionException("Conversion of Markdown markup failed: " + exception.toString()); }

And also for one of our other requirement we are extending LinkRenderer.java of pegdown for Confluence WikiLink Renderer implementation. Can we have a replacement for this one as well?

Currently we are getting compilation errors for Rendering, WikiLinkNodeclasses if we migrate from pegdown to flexmark java.

Please help us resolve these issue would be greatly appreciated. Let me know if anything is not clear.

Thanks RK

vsch commented 7 years ago

@rkanumola, the first part is here:

    static final MutableDataHolder OPTIONS = PegdownOptionsAdapter.flexmarkOptions(
            Extensions.ALL - (headerLinks ? 0 : Extensions.ANCHORLINKS)
                    - (hardwrap ? 0 : Extensions.HARDWRAPS) + (allowHtml ? 0 : Extensions.SUPPRESS_ALL_HTML)
    ).toMutable()
            // set additional options here:
            //.set(HtmlRenderer.FENCED_CODE_LANGUAGE_CLASS_PREFIX,"")
            ;

    static ArrayList<Extension> extensions = new ArrayList<Extension>();

    static {
        // add your extra extensions here
        //extensions.add(ConfluenceWikiLinkExtension.create());
        for (Extension ext : OPTIONS.get(Parser.EXTENSIONS)) {
            extensions.add(ext);
        }

        OPTIONS.set(Parser.EXTENSIONS, extensions);
    }

    static final Parser PARSER = Parser.builder(OPTIONS).build();
    static final HtmlRenderer RENDERER = HtmlRenderer.builder(OPTIONS).build();

    // use the PARSER to parse and RENDERER to render with pegdown compatibility
    static {
        Node document = PARSER.parse(data);
        processed = RENDERER.render(document);
    }

What does your ConfluenceWikiLinkRenderer do that is different from GitHub Wiki links?

You can use the code above which will have wiki links enabled and will convert [[wiki page]] to references.

To have custom link resolving logic added you need a custom link resolver. For example see: PegdownCustomLinkResolverOptions

An overview of parsing and rendering phases with available extension points can be found in the wiki: https://github.com/vsch/flexmark-java/wiki/Writing-Extensions

vsch commented 7 years ago

@rkanumola, sorry forgot I added parameters for extra pegdown options adapter extensions. Shorter version of above:

    static final MutableDataHolder OPTIONS = PegdownOptionsAdapter.flexmarkOptions(
            Extensions.ALL - (headerLinks ? 0 : Extensions.ANCHORLINKS)
                    - (hardwrap ? 0 : Extensions.HARDWRAPS) + (allowHtml ? 0 : Extensions.SUPPRESS_ALL_HTML)
            // add your extra extensions here
            //, new ConfluenceWikiLinkExtension()
    ).toMutable()
            // set additional options here:
            //.set(HtmlRenderer.FENCED_CODE_LANGUAGE_CLASS_PREFIX,"")
            ;

    static final Parser PARSER = Parser.builder(OPTIONS).build();
    static final HtmlRenderer RENDERER = HtmlRenderer.builder(OPTIONS).build();
    // use the PARSER to parse and RENDERER to render with pegdown compatibility
    static {
        Node document = PARSER.parse(data);
        processed = RENDERER.render(document);
    }
rkanumola commented 7 years ago

@vsch , thank you so much for the response,with your suggestion I was able to resolve my issue. I have one more doubt. Currently I am using the following classes in my implementation. in pom.xml if I use flexmark or flexmark-all artifact my total plugin artifact size has been increased from 13.2 MB from 1.6 MB, because of this installing the add-on also taking much time than earlier. am I using the right dependency ? Please suggest.

import com.vladsch.flexmark.ast.Node;
import com.vladsch.flexmark.html.HtmlRenderer;
import com.vladsch.flexmark.parser.Parser;
import com.vladsch.flexmark.profiles.pegdown.Extensions;
import com.vladsch.flexmark.profiles.pegdown.PegdownOptionsAdapter;
import com.vladsch.flexmark.util.options.MutableDataHolder;

Update: I am not sure what happen, wiki links are stopped working all of a sudden it used to work with the above shared code till today morning. Can you have any pointers on that?

The following code which I have integrated in my implementation.

MutableDataHolder OPTIONS = PegdownOptionsAdapter.flexmarkOptions(
                    Extensions.ALL - (headerLinks ? 0 : Extensions.ANCHORLINKS) - (hardwrap ? 0 : Extensions.HARDWRAPS)
                            + (allowHtml ? 0 : Extensions.SUPPRESS_ALL_HTML)).toMutable()
            // set additional options here:
            // .set(HtmlRenderer.FENCED_CODE_LANGUAGE_CLASS_PREFIX,"")
            ;

            ArrayList<Extension> extensions = new ArrayList<Extension>();

            // add your extra extensions here
            // extensions.add(ConfluenceWikiLinkExtension.create());
            for (Extension ext : OPTIONS.get(Parser.EXTENSIONS)) {
                extensions.add(ext);
            }

            OPTIONS.set(Parser.EXTENSIONS, extensions);

            final Parser PARSER = Parser.builder(OPTIONS).build();
            final HtmlRenderer RENDERER = HtmlRenderer.builder(OPTIONS).build();

            // use the PARSER to parse and RENDERER to render with pegdown compatibility
            Node document = PARSER.parse(data);
            processedHtml = RENDERER.render(document);
rkanumola commented 7 years ago

@vsch , can we have your response on the above issues please ?

vsch commented 7 years ago

@rkanumola, the size increase is caused by a dependency on flexmark-pdf-converter module. One of its dependencies, icu4j is 11.6 MB and fontbox is 1.4 MB. This is included in the flexmark-all module but not in flexmark-profile-pegdown.

rkanumola commented 7 years ago

@vsch , thanks for the response on the artifact size related query. Can you please suggest what is the problem in my above mentioned code which is causing Wiki links not to work. Any pointers on this would be greatly appreciated.

vsch commented 7 years ago

@rkanumola, I tested a wiki link in sample code and it is parsing and rendering as expected.

Can you provide a sample of Markdown with Wiki links that is not working? Best if you output the value of the data string that is being parsed.

rkanumola commented 7 years ago

@vsch , I have consolidated my rendering problems which are identified the reason for migrating from pegdown to flexwork.

pegdown-flexmark-migration-issues.docx

Please let me know if anything is not clear or need any other details for your analysis.

rkanumola commented 7 years ago

@vsch , can you please provide your suggestion to move forward with our migration from pegdown to flexmark. We are one of the vendors of Atlassian's Confluence add-ons. Need to deliver the migrated artifact at the earliest as possible. Let me know if you need any other details.

vsch commented 7 years ago

@rkanumola, thank you for the discrepancy document. I am going through the document and will open separate issues for each one I encounter.

As a general comment it would be easier to address the issues if the differences were isolated in a relevant sample. For example, image title tag missing bug in image link reference. The only significant part is:

![alt text][id]

[id]: /images/icons/up_16.gif "Title"

Actual:

<p><img src="/images/icons/up_16.gif" alt="alt text" /></p></div></p>

Expected:

<p><img src="/images/icons/up_16.gif" alt="alt text" title="Title" /></p>

Much easier to understand without all the noise of unrelated markdown and HTML that is in the document. Also the summary of "Missing title tag" to direct attention would help.

### Images ###

Image syntax is very much like link syntax.

Inline (titles are optional):

![alt text](/images/icons/up_16.gif "Title")

Reference-style:

![alt text][id]

[id]: /images/icons/up_16.gif "Title"

Actual:

<p><div class="markdown-macro conf-macro output-inline" data-hasbody="true" data-macro-name="markdown" id="markdown-macro-1"><h3 id="Images">Images</h3>
<p>Image syntax is very much like link syntax.</p>
<p>Inline (titles are optional):</p>
<p><img src="/images/icons/up_16.gif" alt="alt text" title="Title" /></p>
<p>Reference-style:</p>
<p><img src="/images/icons/up_16.gif" alt="alt text" /></p></div></p>

Expected:

<p>
<div id="markdown-macro-1" class="markdown-macro conf-macro output-inline" data-hasbody="true" data-macro-name="markdown">
   <h3 id="Images">Images</h3>
   <p>Image syntax is very much like link syntax.</p>
   <p>Inline (titles are optional):</p>
   <p><img src="/images/icons/up_16.gif" alt="alt text" title="Title" /></p>
   <p>Reference-style:</p>
   <p><img src="/images/icons/up_16.gif" alt="alt text" title="Title" /></p>
</div>
</p>
vsch commented 7 years ago

@rkanumola, my bad on the image ref. Just found that the simple case works but the complex one does not. Debugging it now to see what is causing it.

vsch commented 7 years ago

@rkanumola, #109 updated. Bug confirmed and fixed for next release.

As for Wiki Links the Wiki links extension requires that you add a LinkResolver to handle your link resolution for Wiki links which are implementation dependent. I will add the link and information to the extension docs.

See: PegdownCustomLinkResolverOptions

You will need to proved code that handles Confluence handling of the wiki link content.

vsch commented 7 years ago

@rkanumola, repo updated. Maven released, may take time for central to update.

rkanumola commented 7 years ago

@vsch , thanks for your responses. as request I am here with attached our Confluence Wiki links implementation by using pegdown for your reference. The following class used to call as mentioned in the first message. I will use the latest artifact and give you an update after using it. Thanks again for the work done so far.

PegDownProcessor pegDownProcessor = new PegDownProcessor(Extensions.ALL - (headerLinks ? 0 : Extensions.ANCHORLINKS)

  • (hardwrap ? 0 : Extensions.HARDWRAPS) + (allowHtml ? 0 : Extensions.SUPPRESS_ALL_HTML)); try { processed = pegDownProcessor.markdownToHtml(data, new ConfluenceWikiLinkRenderer(info, xhtmlContent)); } catch (Exception exception) { exception.printStackTrace(); throw new MacroExecutionException("Conversion of Markdown markup failed: " + exception.toString()); }

ConfluenceWikiLinkRenderer.java.txt

rkanumola commented 7 years ago

@vsch , Thanks for #109 fix, I have verified the fix and found it is working as expected now. Regarding the Wiki links and other headerLink issues can I have your pointers please ?. I have shared the required data for wiki link for your debug analysis information in the last comment. Do we have any existing samples which meets our requirement by using LinkResolver ?

vsch commented 7 years ago

@rkanumola, pegdown does not resolve wiki links as you have shown. This is probably done in the ConfluenceWikiLinkRenderer which is custom code in your project.

You need to take the logic from it and convert it to a flexmark-java link renderer for Wiki link nodes. Sample given above PegdownCustomLinkResolverOptions you should add your logic in the if (node instanceof WikiLink) and use the node's information like getLink() to get the link text and parse it according to Confluence wiki rules.

rkanumola commented 7 years ago

Thanks @vsch for your suggestion, I trying to change as much as possible but still not able to get there. Any smart way of converting our custom class to flexmark link render format? Your help is much appreciated in this regard.

vsch commented 7 years ago

@rkanumola, where did you get the confluence renderer and why can't you convert the renderer to flexmark?

Original pegdown library does not come with a confluence renderer so I am assuming it is your custom code.

You can almost copy/paste logic from the custom renderer into the flexmark with the change that in flexmark there is a single method to map URL text to resolved link with the node type used to distinguish the source node while in Pegdown renderer there is a method per node element. The sample has the if () node type tests. The rest is a matter of getting the URL text from the node and applying the logic in your renderer. The result to be returned in the structure as shown in the sample provided.

Which part is giving you difficulty in the conversion?

rkanumola commented 7 years ago

@vsch , Thanks for responding. I am herewith attaching the difficulties I am having in converting our custom class to get the wikilinks work. Please help me in this regard. Let me know if anything is not clear in the document.

Thanks Clarification.docx

vsch commented 7 years ago

@rkanumola, screenshots of the code for your custom confluence renderer is impossible to see in the doc.

The compilation errors are caused by pegdown API references. When you migrate the logic of the class to a flexmark custom node resolver then the issue will be resolved.

  1. Wiki links will work when you implement your custom renderer as flexmark custom link resolver to replace the confluence renderer.

  2. As for exceptions during parsing, I cannot see the code but would say that flexmark parsing only generates exceptions in the case of internal bugs. Markdown is always valid and does not generate parsing errors. There is also no timeout exceptions in flexmark as there is in pegdown.

  3. I can see that regex is used to parse the wiki link text which is what I expected. You need to extract this code and add it to the custom link resolver sample so that it achieves the same effect. Replace the new Rendering() by the link.withURL() and other helper functions of the ResolvedLink class to return the resolved URL as in the sample. Also change the node.getText() from pegdown's API to node.getChars() or link.getUrl() to get the wiki link URL only but in the latter case you will need to change the parsing logic because the URL does not include the [[ and ]] of the wiki link.

In the future could you please copy/paste the valid information into the issue directly, rather than include a separate document. That way it will be easier to follow. You can paste images directly into comments or drop them as files.

rkanumola commented 7 years ago

@vsch , thank you so much for the response. I am applying your suggestions, will keep you updated.

rkanumola commented 7 years ago

@vsch, I am getting few issue while replacing my customwiki link logic to our flexmark LinkResolver. Problem is that in my customwikilink implementation I was able to pass XhtmlContent and MacroInfo as an arguments with the render override method of LinkRenderer Pegdown API, but in our case I am not able to instantiate, which are most important parameters in my custom implementation where I will pass those two parameters to my utility method(ScriptUtil) which will return me the rendered href wiki link. i.e converting storage format hyper link </ac:link> to actual anchor tag to use i.e a href="/display/markdown/Home">home [Removed < tag intentionall so that it will not render as a link] I am trying to give you the existing implementation and current implementation as per your suggestion. Please take a look into it and give your inputs on it. Please let me know if anything is not clear.

Existing Custom Wiki Link Implementation:

`public class ConfluenceWikiLinkRenderer extends LinkRenderer {

protected final MacroInfo info;
protected final XhtmlContent xhtmlContent;

// Regex for Confluence wiki link
// string will be of the form: name|space:page^attachment
// label|, space: and ^attachment are all optional
// group 1 is label or space:page (if no label is provided)
// gouup 2 is label indicator
// group 3 is space:page if a label is provided, otherwise blank
// group 4 is attachment
protected static final Pattern CONFLUENCE_WIKI_LINK = Pattern.compile("([^\\|\\^]*)(\\|{0,1})([^\\^]*)\\^{0,1}(.*)"); // handle xxx or xxx|
protected static final Pattern SPACE_PAGE = Pattern.compile("([^\\:]*)\\:(?!/)(.*)"); // handle xxx:yyy but not http://zzz

protected static final Pattern HTML_LINK = Pattern.compile("<a\\s[^>]*href\\s*=\\s*\"([^\"]*)\"[^>]*>(.*?)</a>");

public ConfluenceWikiLinkRenderer(final MacroInfo info, final XhtmlContent xhtmlContent) {
    this.info = info;
    this.xhtmlContent = xhtmlContent;
}

/**
 * See if we can match on specific Confluence wiki link format. Otherwise default to pegdown behavior
 * 
 * @see org.pegdown.LinkRenderer#render(org.pegdown.ast.WikiLinkNode).
 */
@Override
public Rendering render(WikiLinkNode node) {
    final Matcher matcher = CONFLUENCE_WIKI_LINK.matcher(node.getText());
    // log.debug("text: {}", node.getText());

    if (matcher.find()) {
        try {
            log.debug("group1: {}, group2: {}", matcher.group(1), matcher.group(2));
            log.debug("group3: {}, group4: {}", matcher.group(3), matcher.group(4));

            String label = "";
            String page;
            if (StringUtils.isBlank(matcher.group(2))) { // no label provided - indicator is missing
                page = matcher.group(1);
            } else {
                label = matcher.group(1);
                page = matcher.group(3);
            }
            log.debug("page: {}", page);

            String spaceKey = "";
            String spaceString = "";

            if (!StringUtils.isBlank(page)) {  // page is not blank
                final Matcher matcherSpacePage = SPACE_PAGE.matcher(page);
                if (matcherSpacePage.find()) {
                    spaceKey = matcherSpacePage.group(1).trim();
                    spaceString = "ri:space-key=\"" + spaceKey + "\" ";
                    page = matcherSpacePage.group(2).trim(); // reset page to the page portion of space:page
                } else if (page.contains("://")) {
                    return new Rendering(page, matcher.group(1)); // just render a regular link
                }
            }

            String attachmentString = "";
            String attachmentStringEnd = "";

            String spaceStartString = page.equals("") && !spaceKey.equals("") ? "<ri:space " : "";

            if (!StringUtils.isBlank(matcher.group(4))) { // attachment defined
                // Attachment attachment = attachmentManager.getAttachment(info.getContent(), attachmentString); // if we need to validate attachment

                // Example: <ac:link><ri:attachment ri:filename="exploit.csv"><ri:page ri:content-title="a"
                // ri:space-key="EXPERIMENT"/></ri:attachment></ac:link>
                // Example: no space or page
                // <ac:link><ri:attachment ri:filename="basics.text" /><ac:plain-text-link-body><![CDATA[aaa]]></ac:plain-text-link-body></ac:link>

                attachmentString = "<ri:attachment ri:filename=\"" + GeneralUtil.htmlEncode(matcher.group(4)) + "\"> ";
                attachmentStringEnd = "</ri:attachment>";
                log.debug("attachment: {}, label: >>>{}<<<", matcher.group(4), label);

                spaceStartString = "";
            }
            log.debug("spaceString: {}", spaceString);

            String pageString = spaceStartString + (StringUtils.isBlank(page) ? "" : "<ri:page ri:content-title=\"" + GeneralUtil.htmlEncode(page) + "\" ");
            String plainTextString = (StringUtils.isBlank(label) ? "" : "<ac:plain-text-link-body><![CDATA[" + label + "]]></ac:plain-text-link-body>");

            String link = "<ac:link>" + attachmentString + pageString + spaceString + "/>" + attachmentStringEnd + plainTextString + "</ac:link>";

            try {
                final String rendered = ScriptUtils.render(xhtmlContent, info, link);
                log.debug("Link: {}, rendered link: {}", link, rendered);

                final Matcher matcherHtmlLink = HTML_LINK.matcher(rendered);
                if (matcherHtmlLink.matches()) {
                    return new Rendering(matcherHtmlLink.group(1), matcherHtmlLink.group(2));
                } else {
                    return new Rendering("", rendered); // default if we can't match properly
                }
            } catch (MacroExecutionException exception) {
                log.error("Error rendering link: {}", exception.toString());  // this shouldn't happen
                return new Rendering("", "");
            }
        } catch (Exception exception) { // shouldn't happen, but helps for debugging
            log.error("Unexpected error constructing link for " + node.getText(), exception);
            return new Rendering("", "");
        }
    } else {
        return super.render(node); // do whatever the default is
    }
}

}`

Current Implementation As per your suggestion - Yet to finish Calling the custom class like below `MutableDataHolder OPTIONS = PegdownOptionsAdapter .flexmarkOptions( Extensions.ALL - (headerLinks ? 0 : Extensions.ANCHORLINKS) - (hardwrap ? 0 : Extensions.HARDWRAPS)

And its implementation so far as ..

static class CustomExtension implements HtmlRenderer.HtmlRendererExtension {
        @Override
        public void rendererOptions(final MutableDataHolder options) {

        }

        @Override
        public void extend(final HtmlRenderer.Builder rendererBuilder, final String rendererType) {
            rendererBuilder.linkResolverFactory(new CustomLinkResolver.Factory());
        }

        static CustomExtension create() {
            return new CustomExtension();
        }
    }

    static class CustomLinkResolver implements LinkResolver {

        public CustomLinkResolver(final NodeRendererContext context) {
            // can use context for custom settings
            // context.getDocument();
            // context.getHtmlOptions();
        }

        @Override
        public ResolvedLink resolveLink(final Node node, final NodeRendererContext context, final ResolvedLink link) {
            final Pattern CONFLUENCE_WIKI_LINK = Pattern.compile("([^\\|\\^]*)(\\|{0,1})([^\\^]*)\\^{0,1}(.*)"); // handle xxx or xxx|
            final Pattern SPACE_PAGE = Pattern.compile("([^\\:]*)\\:(?!/)(.*)"); // handle xxx:yyy but not http://zzz
            if (node instanceof WikiImage) {
                // resolve wiki image link
                String url = link.getUrl() + ".png";

                // resolve url, return one of LinkStatus other than LinkStatus.UNKNOWN
                return link.withStatus(LinkStatus.VALID).withUrl(url);
            } else if (node instanceof WikiLink) {
                // resolve wiki link
                String url = link.getUrl();
                Matcher matcher = CONFLUENCE_WIKI_LINK.matcher(node.getChars());
                if (matcher.find()) {

                    System.out.println("group1: {}, group2: {}" + matcher.group(1) + "\b" + matcher.group(2));
                    System.out.println("group3: {}, group4: {}" + matcher.group(3) + "\b" + matcher.group(4));
                }
                String label = "";
                String page;
                if (StringUtils.isBlank(matcher.group(2))) { // no label provided - indicator is missing
                    page = matcher.group(1);
                } else {
                    label = matcher.group(1);
                    page = matcher.group(3);
                }

                String spaceKey = "";
                String spaceString = "";
                if (!StringUtils.isBlank(page)) {  // page is not blank
                    final Matcher matcherSpacePage = SPACE_PAGE.matcher(page);
                    if (matcherSpacePage.find()) {
                        spaceKey = matcherSpacePage.group(1).trim();
                        spaceString = "ri:space-key=\"" + spaceKey + "\" ";
                        page = matcherSpacePage.group(2).trim(); // reset page to the page portion of space:page
                    } else if (page.contains("://")) {
                        return link.withUrl(matcher.group(1)); // just render a regular link
                    }
                }

                String attachmentString = "";
                String attachmentStringEnd = "";

                String spaceStartString = page.equals("") && !spaceKey.equals("") ? "<ri:space " : "";

                if (!StringUtils.isBlank(matcher.group(4))) { // attachment defined

                    attachmentString = "<ri:attachment ri:filename=\"" + GeneralUtil.htmlEncode(matcher.group(4)) + "\"> ";
                    attachmentStringEnd = "</ri:attachment>";

                    spaceStartString = "";
                }

                String pageString = spaceStartString + (StringUtils.isBlank(page) ? "" : "<ri:page ri:content-title=\"" + GeneralUtil.htmlEncode(page) + "\" ");
                String plainTextString = (StringUtils.isBlank(label) ? "" : "<ac:plain-text-link-body><![CDATA[" + label + "]]></ac:plain-text-link-body>");

                String link1 = "<ac:link>" + attachmentString + pageString + spaceString + "/>" + attachmentStringEnd + plainTextString + "</ac:link>";
                System.out.println(link1);

                **final String rendered = ScriptUtils.render(xhtmlContent, info, link);** // Getting problem here .
                // resolve url, return one of LinkStatus other than LinkStatus.UNKNOWN
                return link.withStatus(LinkStatus.VALID).withUrl(url);
            }
            return link;
        }

        static class Factory implements LinkResolverFactory {
            @Override
            public Set<Class<? extends LinkResolverFactory>> getAfterDependents() {
                return null;
            }

            @Override
            public Set<Class<? extends LinkResolverFactory>> getBeforeDependents() {
                return null;
            }

            @Override
            public boolean affectsGlobalScope() {
                return false;
            }

            @Override
            public LinkResolver create(final NodeRendererContext context) {
                return new CustomLinkResolver(context);
            }
        }
    }

I am also attaching sample outputs from pegdown to flexmark migration as of now for wikilink for your reference.

after.html.txt before.html.txt

sunitapatro commented 7 years ago

Hi @vsch, I work with @rkanumola. Thanks much for your help so far! Just to make it short, we need a way to pass parameters to CustomLinkResolver() for our custom code to work. Eg: We were passing info, xhtmlContent to ConfluenceWikiLinkRenderer as below, similarly how to pass these parameters to CustomLinkResolver? processed = pegDownProcessor.markdownToHtml(data, new ConfluenceWikiLinkRenderer(info, xhtmlContent));

vsch commented 7 years ago

@sunitapatro, you can pass any information you want through options DataHolder instance used to create Parser and HtmlRenderer.

If you re-use the parser and html renderer instances and need to change options for each invokation of html renderer, you can pass it using the Document node, which is a MutableDataHolder.

Define your own DataKey<T>:

public static final DataKey<XhtmlContent> XHTML_CONTENT = new DataKey<XhtmlContent>("XHTML_CONTENT", (XhtmlContent)null);

Then pass it in the Document node before rendering the document:

Document document = (Document) parser.parse(markdown);
document.set(XHTML_CONTENT, xhtmlContent);
String html = renderer.render(document);

In your link resolver or anywhere in the rendering process you can get this data through the document node:

public ResolvedLink resolveLink(final Node node, final NodeRendererContext context, final ResolvedLink link) {
    Document document = context.getDocument();
    XhtmlContent xhtmlContent = document.get(XhtmlContent);
    ...;
}

A sample source of this is available at CustomContextDataSample

rkanumola commented 7 years ago

@vsch , we are almost done with the migration with the continuous support from you. But we are having two queries/problems as of now which needs your inputs.

For the below input we are observing the different output when compare to pegdown and flexmark. [[home]] [[zmarkdown:home]] [[examples:Home]] -- goes to examples home [[aaa|zmarkdown:home]] [[zmarkdown:]] [[bbb|zmarkdown:]] [[http://google.com]] [[zzz|http://google.com]]

Attached the resultant values for your reference, we would like to know is there anything missing from end which is resulting in the different anchor tag text?

Another query is that when we give the input for the special characters For Eg : 'Wiki links - with special characters " \' # % < > ! @'

We always see that single quote always converting Single Quote as an Apostrophe. Please check this issue as well.

Flexmark Output flexmark_output

Pegdown Output pegdown_output

Thanks in advance.

vsch commented 7 years ago

@rkanumola, it appears that you need to change the link text for wiki links. You need to add a custom link renderer for wiki links.

You can make a copy of the renderer from WikiLink extension and modify it to handle wiki link text changes.

As for the single quote to apostrophe change, this is done by the pegdown Extensions.SMARTS flag. It beautifies single quotes, ellipsis, n-dash, m-dash sequences.

rkanumola commented 7 years ago

@vsch , regarding custom link renderer for wiki links we have implemented as per your suggestions followed in CustomContextDataSample, looking for whether equivalent pegdown output is available in flexmark as mentioned above or this is how the flexmark will return the output ,would like to clarify.

I will try out Extensions.SMARTS option and update you on that.

vsch commented 7 years ago

@rkanumola, actually more thought about the typographic extension and wiki links leads me to think that converting single quote to apostrophe is a bug wiki link renderer should convert the wiki link text to non-typographic original characters before using the text to derive the page reference text.

Wiki link text is both a page reference and link text. In this case the typographic extension converted text should be used for the link text but the original should be used for the page reference to make the rendering make sense.

I will make a fix for this.

The rest of the behaviour is as expected. I don't understand why under pegdown [[zmarkdown:]] renders as if it was [[zmarkdown]] and [[zmarkdown:home]] renders as if it was [[zmarkdown:home]]. If this is the desired behaviour then your custom wiki link renderer should add the logic to modify the link text for these conditions.

vsch commented 7 years ago

@rkanumola, I double checked and the current implementation already handles the typographic conversion only for the link text part, not the page ref part.

To disable this conversion for the link text you should remove the Extensions.SMARTS and Extensions.QUOTES flags from your pegdown options.