TOC support

[chaopeng]

use [TOC] in markdown and using PegDownProcessor with Extensions.TOC can generate the Table of Contents. also support using [TOC level=n] n=[1,6] to limit the max TOC level.

[vsch]

@chaopeng thank you for the extension and the tests.

Some comments:

1. Including the TOC flag as an optional extension that is included in Extensions.ALL breaks half the existing tests. Either these tests need to be updated to pass with the new extension or the TOC extension should be added as one that is not included in Extensions.ALL
2. As it is implemented, this extension is not compatible with EXTANCHORLINKS extension, when both are used the TOC extension generates empty id's and empty list text because with EXTANCHORLINKS the text for the anchor link does not come from the first child of the header node but from all the children.

If you could address these it would be great. Otherwise, I will look into it as soon as I get the chance.

@sirthias, it will require rework to work with EXTANCHORLINKS and reworking existing tests is going to be a pain, on my branch 27 of 53 tests fail. I would probably opt out to make it an optional extension not included in ALL to avoid the extra re-work of tests.

I will try to spend some time on it since it is a nice addition but I am working on a major release of the plugin and will not be able to get to it for at least a week.

[chaopeng]

@vsch thank you. I changed the Extensions.TOC as

    /**
     * Enables TOC
     */
    static final int TOC = 0x00800000;

and will has 1 fail in Markdown103Spec same as master now.

[chaopeng]

@vsch One thing i forgot, if the <h>not english</h> will generate a valid id.

[marcelstoer]

Great! I hope this PR lands soon. TOC support is absolutely essential but so many Markdown implementations have no support for it.

[denisftw]

Any update on this?

[chaopeng]

@denisftw i don't know. I can work on this if they have plan.

[vsch]

@chaopeng, I fixed a bug in HTML rendering of TOC when some intervening header levels are missing. The code does not open a <li> tag for these but at the end does close it.

My pegdown version is too different from the master to make a PR and I am replacing pegdown with a parser I rewrote from commonmark-java, called flexmark-java to have a pegdown compatible mode and extensions. It parses 20x-25x faster than pegdown and without the pathological input hangups. Easy to extend the grammar and add extensions. All this to say that I can't PR but here is the modified version of the function if you are interested in updating your PR.

    public void visit(TocNode node) {
        if (!node.getHeaders().isEmpty()) {
            int initLevel = node.getHeaders().get(0).getLevel();
            int lastLevel = node.getHeaders().get(0).getLevel();
            boolean[] openedItems = new boolean[7];

            printer.println().print("<ul>").println();

            for (int i = 0; i < node.getHeaders().size(); ++i) {
                HeaderNode header = node.getHeaders().get(i);
                int headerLevel = header.getLevel();

                // ignore the level less than toc limit
                if (headerLevel > node.getLevel()) {
                    continue;
                }

                if (lastLevel < headerLevel) {
                    for (int lv = lastLevel; lv < headerLevel; ++lv) {
                        printer.print("<ul>").println();
                        openedItems[lv+1] = false;
                    }
                } else if (lastLevel == headerLevel) {
                    if (i != 0) {
                        printer.print("</li>").println();
                        openedItems[lastLevel] = false;
                    }
                } else {
                    printer.print("</li>").println();
                    for (int lv = lastLevel; lv > headerLevel; lv--) {
                        printer.print("</ul></li>").println();
                        openedItems[lv] = false;
                    }
                }

                printer.print("<li><a href=\"#").print(header.getId()).print("\">").print(header.getText()).print("</a>");
                openedItems[headerLevel] = true;
                lastLevel = headerLevel;
            }

            for (int i = lastLevel; i >= initLevel; i--) {
                if (openedItems[i]) printer.print("</li>");
                printer.print("</ul>").println();
            }

            printer.println();
        }
    }

[chaopeng]

OK, I will do it next weekend. Happy Canada day.