Only Unix Lineend supported

[Hoffenbar]

If I write a document with CRLF line endings (as is usual in windows), then the string <!-- toc --> isn't found, maybe because you are comparing against <!-- toc -->\n and \n really seems to be "Newline" only in Rust. In some languages, like Tcl, \n normally matches CRLF (Windows), CR (Mac), or LF (*Nix), so that wouldn't be a concern there. It would be nice, if every common line structure would be possible.

[badboy]

We can match against the trimmed string and that should do it.

[mlange-42]

A workaround I just found while trying to solve an issue caused by this behaviour is to set the marker in the book.toml:

[preprocessor.toc]
command = "mdbook-toc"
marker = "<!-- toc -->"
renderer = ["html"]

The problem is that the default marker is defined as

static DEFAULT_MARKER: &str = "<!-- toc -->\n";

EDIT: I realized that this workaround only generates a TOC when using Windows line endings, but not with Unix line endings.

[mlange-42]

After some research in mdbook-toc and pulldown-cmark code, I see the primary problem in comparing pulldown-cmark Events to find the TOC pattern, in combination with the fact that pulldown-cmark adds breaks on \n as well as \r.

I was wondering if it is really necessary to parse with pulldown-cmark to find to TOC pattern. Wouldn't it be sufficient to simply search for the pattern using String::find (or RegEx) and replace the match by the TOC?

Sure, parsing is necessary to find headings to be included in the TOC, I am just talking about finding the pattern to be replaced.

[badboy]

That was a recent new addition. The marker now supports multiline markers such as the following too:

* auto-gen TOC;
{:toc}

That's why the code now parses the marker itself as markdown too.

As I personally don't have the use for parsing anything other that \n as newline in that marker I unfortunately can't help much more at the moment.

[mlange-42]

How about detecting line endings on a per-file-basis and modifying the pattern accordingly?

I would be happy to give it a try (with my still limited Rust experience) and prepare a merge request.

[mlange-42]

@Hoffenbar I found that using markers that are not interpreted as HTML seem to work with both kinds of line endings, e.g.

marker = "[[_TOC_]]"

Maybe that helps as a workaround. Of course, the marker will then not be hidden in the unprocessed Markdown documents as the HTML comment marker would.

EDIT: But see also #13

[badboy]

With #13 fixed and markers such as [[_TOC_]] working I think we can close this now.