Options for custom jinja comment tag markers

[foenixx]

Hi,

In my docs there are lots of {# .. }. Jinja treats them as invalid comments and fails. Is it possible to provide options for custom comment tag markers, in addition to existing j2_block... and j2_variable... options?

[github-actions[bot]]

Welcome to this project and thank you!' first issue

[fralau]

Oops, this is a delicate question. The answer is "no, not at this moment".

I remember there was an issue that forced me to slightly redefine the syntax of Jinja2 comments (I don't remember exactly what). In principle, adding a parameter to the config file for custom marker should not be a big deal; as long as it doesn't break everything else 🙂.

I need to check on this and get back to you.

[honzajavorek]

I just found this section in the docs: https://mkdocs-macros-plugin.readthedocs.io/en/latest/advanced/#how-to-prevent-interpretation-of-jinja-like-statements Wouldn't any of the solutions listed be sufficient to prevent the problem?

[foenixx]

I've read it. All proposed methods look clumsy and I don't want to introduce a lot of new text or complex syntax constructions in my markdown. For now I've fixed the problem by inserting so-called "word joiner" unicode character between { and # . But this solution has it's flip side also )).

[fralau]

@foenixx

I've read it. All proposed methods look clumsy and I don't want to introduce a lot of new text or complex syntax constructions in my markdown.

I understand. If possible, could you explain briefly what is the reason why you have all these comments in your markdown/jinja2 code? Perhaps that would give me an idea "out of the box".

[foenixx]

If possible, could you explain briefly what is the reason why you have all these comments in your markdown/jinja2 code?

We have a lot of documentation of a huge enterpise ECM-BMP-CMS system and it's written in asciidoc. Asciidoc has lots of drawbacks for us and we decided to migrate to markdown (mkdocs + mkdocs-material). A part of this system has it's own simple DSL and it uses {# ... } constructions heavily. They are scattered all over the docs )).

I developed a converter from asciidoc to markdown and this issue is one of a billion of small issues I have to fix ))). Actually it's not a big deal, since I've fixed it in converter, so you can just close the issue.

[fralau]

@foenixx Wow, this sounds like a very exciting project, and full of challenges, too! 👏

I am sure that the conversion process is something you already had a good look at...

To convert from asciidoc to markdown en masse, I would intuitively look on the side of the pandoc document converter. It's a wonderful utility for that kind of things. Pandoc understands both asciidoc and markdown.

This is never going to work perfectly out of the box, of course. But if the project is big enough, it might warrant writing some of your own, ad-hoc filters. Since you will work with Python, have you had a look at Sergio Correia's planflute?

[foenixx]

Wow, this sounds like a very exciting project, and full of challenges, too! 👏 Yes, indeed )).

The most exciting ones are:

• Cross-document links.
• Some links refer to generated html instead of adoc source.
• Some links refer to auto-generated permanent links.
• Heavy use of asciidoc complex tables with nested paragraphs and syntax blocks. Those are to be converted into markdown lists since markdown tables are pretty simple.
• Detecting simple tables without complex data in the cells so they can be left as is and converted into markdown tables.
• Lots of quirks in text. Complex and exquisite formatting.
• Convering asciidoc admonitions into markdown admonitions.
• Huge files. They are to be split into smaller ones by 2-level headers. Don't forget there could be links to them.
• Lists nested in lists nested in lists...
• Includes )).

I tried pandocs and the results where not impressing. So I decided to develop a converter of my own and at the moment it works fine (mostly). 😄

[fralau]

Nice little project...

I tried pandocs and the results where not impressing. So I decided to develop a converter of my own and at the moment it works fine (mostly). 😄

Yes, I'm not suprised, that pandoc's result were not impressive, if you used it as a utility out of the box.

Things would be different, if pandoc is used as part of a toolkit. Pandoc has its own internal "pivot" AST (independent from the language used), which can be a tremendous asset.

It could work for you only if you found the proper pandoc filters, or you wrote your own. Filters are basically programs that work directly on the AST (exactly what you are describing: complex tables, inter-document links, etc.).

For the most horrible formatting quirks (weird asciidoc constructs that pandoc simply ignores), you might want to write your own "pre-processor" to predigest them.

I've done the whole drill to convert Word documents into LaTeX, via Jinja2-enriched markdown, so I have subjective experience of it. I know this can be done.

But regardless of whether you use pandoc or not, there is a certainty: all features of asciidoc that could not be translated into standard markdown or a markdown extension, you might have to translate into jinja2 macros that emulate the original asciidoc syntax/semantics.

If that's the essential reason why you are looking at mkdocs-macros, you are in the right place 🙂 .

If you want explore further, I would also be willing continue the exchange in a git Discussion (or privately).

[foenixx]

I wish i'd known it when I started two weeks ago... ))) By now I overcame most of the troubles and my converter works reasonably well. But it's quite interesting to know all that stuff about pandocs. Next time I will certainly give it a try.

[fralau]

@foenixx I personally made a similar experience with that subject: personally, I didn't know about Jinja2, and had to go through the whole Evolution again: first the m4 macro processor and then the pyexpander macro processor (plus the experience of wiki macros, 10-15 years ago). So when I finally arrived at Jinja2, it made a huge amount of sense.

Looking back, I find that the detour by the hard and rocky path was not at all a waste of time. It provided valuable challenges, insights and teachings. And as soon as I found an opportunity, I was back to the high road and running at full speed. So fast, actually, that I quickly left others behind 😄. Which is why I wanted to share mkdocs-macros with the Community, as one example of what could be achieved by merging jinja2 and markdown (jinja2 added to markdown, is like nitrous oxide added to the engine of a stock car...).

I wish you all the best in you trek. 🙂 Next time I see a 🚀 flying by, perhaps it will be you...

[fralau]

@foenixx How is it going?