search

fralau / mkdocs-macros-plugin

Create richer and more beautiful pages in MkDocs, by using variables and calls to macros in the markdown code.
https://mkdocs-macros-plugin.readthedocs.io
Other
333 stars 51 forks source link

Incompatibility with d2 plugin in version 1.3 #249

Closed Karreg closed 1 day ago

Karreg commented 6 days ago

Hello,

I have found an incompatibility with d2 plugin in the 1.3 version, that did not exist in version 1.2. I don't have all the details, but here is what I found:

Since version 1.3, I get this error when building my documentation:

Error: The current file is not set for the '!relative' tag. It cannot be used in this context; the intended usage is within `markdown_extensions`.

Without more context in the error, I could not find out which content generated this error, however:

My guess is that since version 1.3, macros plugin analyzes incorrectly the d2 generated content, which is SVG. But that is just a guess.

I have created a repository (with a devcontainer if that can help) here to show the issue. The mkdocs content is minimal (no content other than a simple d2 diagram, no plugin other than search, macros and d2, no theming...): https://gitlab.com/karreg/mkdocs-macros-d2-issue

If I can help, I'll be glad to try :)

github-actions[bot] commented 6 days ago

Welcome to this project and thank you!

fralau commented 4 days ago

Thanks! I had a look at the example, which is quite straightforward.

Offhand, I can't see a logical explanation why this change of version in Mkdocs-Macros is creating a problem. Let's find out.

  1. Could you give the error message you get?
  2. Have you thought of running the serve/build process in verbose mode (-v)?

I am not fully familiar with te d2 project., though it obviously does something similar to the mkdocs-mermaid2 plugin, except for the fact that it is rendering the diagram at build/serve time (am I correct?).

Karreg commented 3 days ago

Hello,

To answer both questions, the error message I get in the build process in verbose mode is the message in the opening issue. No more, no less.

However, here is the full build stack in verbose mode for the record:

$ mkdocs build -v
DEBUG   -  Loading configuration file: /workspaces/mkdocs-macros-d2-issue/mkdocs.yml
DEBUG   -  Loading configuration file: /workspaces/mkdocs-macros-d2-issue/mkdocs.yaml
DEBUG   -  Loaded theme configuration for 'mkdocs' from '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/themes/mkdocs/mkdocs_theme.yml':
           {'static_templates': ['404.html'], 'locale': 'en', 'include_search_page': False, 'search_index_only': False, 'highlightjs': True, 'hljs_languages':
           [], 'hljs_style': 'github', 'hljs_style_dark': 'github-dark', 'navigation_depth': 2, 'nav_style': 'primary', 'color_mode': 'light',
           'user_color_mode_toggle': False, 'analytics': {'gtag': None}, 'shortcuts': {'help': 191, 'next': 78, 'previous': 80, 'search': 83}}
DEBUG   -  Config value 'config_file_path' = '/workspaces/mkdocs-macros-d2-issue/mkdocs.yaml'
DEBUG   -  Config value 'site_name' = 'Incompatibility'
DEBUG   -  Config value 'nav' = [{'Home': 'index.md'}]
DEBUG   -  Config value 'pages' = None
DEBUG   -  Config value 'exclude_docs' = None
DEBUG   -  Config value 'draft_docs' = None
DEBUG   -  Config value 'not_in_nav' = None
DEBUG   -  Config value 'site_url' = None
DEBUG   -  Config value 'site_description' = None
DEBUG   -  Config value 'site_author' = None
DEBUG   -  Config value 'theme' = Theme(name='mkdocs', dirs=['/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/themes/mkdocs',
           '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/templates'], static_templates={'404.html', 'sitemap.xml'}, name='mkdocs',
           locale=Locale('en'), include_search_page=False, search_index_only=False, highlightjs=True, hljs_languages=[], hljs_style='github',
           hljs_style_dark='github-dark', navigation_depth=2, nav_style='primary', color_mode='light', user_color_mode_toggle=False, analytics={'gtag': None},
           shortcuts={'help': 191, 'next': 78, 'previous': 80, 'search': 83})
DEBUG   -  Config value 'docs_dir' = '/workspaces/mkdocs-macros-d2-issue/docs'
DEBUG   -  Config value 'site_dir' = '/workspaces/mkdocs-macros-d2-issue/site'
DEBUG   -  Config value 'copyright' = None
DEBUG   -  Config value 'google_analytics' = None
DEBUG   -  Config value 'dev_addr' = _IpAddressValue(host='127.0.0.1', port=8000)
DEBUG   -  Config value 'use_directory_urls' = True
DEBUG   -  Config value 'repo_url' = 'https://gitlab.com/karreg/mkdocs-macros-d2-issue'
DEBUG   -  Config value 'repo_name' = 'mkdocs-macros-d2-issue'
DEBUG   -  Config value 'edit_uri_template' = None
DEBUG   -  Config value 'edit_uri' = 'edit/master/docs/'
DEBUG   -  Config value 'extra_css' = []
DEBUG   -  Config value 'extra_javascript' = []
DEBUG   -  Config value 'extra_templates' = []
DEBUG   -  Config value 'markdown_extensions' = ['toc', 'tables', 'fenced_code']
DEBUG   -  Config value 'mdx_configs' = {}
DEBUG   -  Config value 'strict' = False
DEBUG   -  Config value 'remote_branch' = 'gh-pages'
DEBUG   -  Config value 'remote_name' = 'origin'
DEBUG   -  Config value 'extra' = {}
DEBUG   -  Config value 'plugins' = {'search': <mkdocs.contrib.search.SearchPlugin object at 0xfaa1eea1b2b0>, 'macros': <mkdocs_macros.plugin.MacrosPlugin
           object at 0xfaa1eea1ba00>, 'd2': <d2.plugin.Plugin object at 0xfaa1ee8e3bb0>}
DEBUG   -  Config value 'hooks' = {}
DEBUG   -  Config value 'watch' = []
DEBUG   -  Config value 'validation' = {'nav': {'omitted_files': 20, 'not_found': 30, 'absolute_links': 20}, 'links': {'not_found': 30, 'absolute_links': 20,
           'unrecognized_links': 20, 'anchors': 20}}
DEBUG   -  Running `config` event from plugin 'search'
DEBUG   -  Running `config` event from plugin 'macros'
INFO    -  [macros] - Configuring the macros environment...
INFO    -  [macros] - Macros arguments
            {'module_name': 'main', 'modules': [], 'render_by_default': True, 'force_render_paths': '', 'include_dir': '', 'include_yaml': [],
           'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'j2_comment_start_string': '',
           'j2_comment_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
DEBUG   -  [macros] - Project dir '/workspaces/mkdocs-macros-d2-issue'
INFO    -  [macros] - No default module `main` found
DEBUG   -  [macros] - Docs directory: /workspaces/mkdocs-macros-d2-issue/docs
DEBUG   -  [macros] - Includes directory: /workspaces/mkdocs-macros-d2-issue/docs
DEBUG   -  [macros] - Undefined behavior: <class 'jinja2.runtime.DebugUndefined'>
INFO    -  [macros] - End of environment config
DEBUG   -  Running `config` event from plugin 'd2'
INFO    -  mkdocs-d2-plugin: Using cache at .cache/plugin/d2/db (dbm.ndbm)
DEBUG   -  Running `pre_build` event from plugin 'search'
DEBUG   -  Running `pre_build` event from plugin 'macros'
INFO    -  [macros] - Config variables: ['extra', 'config', 'environment', 'plugin', 'git', 'macros', 'filters', 'filters_builtin']
DEBUG   -  [macros] - json: cannot encode <class 'mkdocs.theme.Theme'>
DEBUG   -  [macros] - json: cannot encode <class 'method'>
DEBUG   -  [macros] - json: cannot encode <class 'method'>
DEBUG   -  [macros] - json: cannot encode <class 'mkdocs.utils.yaml.RelativeDirPlaceholder'>
Error: The current file is not set for the '!relative' tag. It cannot be used in this context; the intended usage is within `markdown_extensions`.

About the d2-plugin, yes. It builds diagrams from code, in d2 language, which is an alternative to mermaid. It generates SVG as output during the build phase. :)

For the comparison, here is the verbose build stack with macros in version 1.2.0:

$ mkdocs build -v
DEBUG   -  Loading configuration file: /workspaces/mkdocs-macros-d2-issue/mkdocs.yml
DEBUG   -  Loading configuration file: /workspaces/mkdocs-macros-d2-issue/mkdocs.yaml
DEBUG   -  Loaded theme configuration for 'mkdocs' from '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/themes/mkdocs/mkdocs_theme.yml':
           {'static_templates': ['404.html'], 'locale': 'en', 'include_search_page': False, 'search_index_only': False, 'highlightjs': True, 'hljs_languages':
           [], 'hljs_style': 'github', 'hljs_style_dark': 'github-dark', 'navigation_depth': 2, 'nav_style': 'primary', 'color_mode': 'light',
           'user_color_mode_toggle': False, 'analytics': {'gtag': None}, 'shortcuts': {'help': 191, 'next': 78, 'previous': 80, 'search': 83}}
DEBUG   -  Config value 'config_file_path' = '/workspaces/mkdocs-macros-d2-issue/mkdocs.yaml'
DEBUG   -  Config value 'site_name' = 'Incompatibility'
DEBUG   -  Config value 'nav' = [{'Home': 'index.md'}]
DEBUG   -  Config value 'pages' = None
DEBUG   -  Config value 'exclude_docs' = None
DEBUG   -  Config value 'draft_docs' = None
DEBUG   -  Config value 'not_in_nav' = None
DEBUG   -  Config value 'site_url' = None
DEBUG   -  Config value 'site_description' = None
DEBUG   -  Config value 'site_author' = None
DEBUG   -  Config value 'theme' = Theme(name='mkdocs', dirs=['/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/themes/mkdocs',
           '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/templates'], static_templates={'sitemap.xml', '404.html'}, name='mkdocs',
           locale=Locale('en'), include_search_page=False, search_index_only=False, highlightjs=True, hljs_languages=[], hljs_style='github',
           hljs_style_dark='github-dark', navigation_depth=2, nav_style='primary', color_mode='light', user_color_mode_toggle=False, analytics={'gtag': None},
           shortcuts={'help': 191, 'next': 78, 'previous': 80, 'search': 83})
DEBUG   -  Config value 'docs_dir' = '/workspaces/mkdocs-macros-d2-issue/docs'
DEBUG   -  Config value 'site_dir' = '/workspaces/mkdocs-macros-d2-issue/site'
DEBUG   -  Config value 'copyright' = None
DEBUG   -  Config value 'google_analytics' = None
DEBUG   -  Config value 'dev_addr' = _IpAddressValue(host='127.0.0.1', port=8000)
DEBUG   -  Config value 'use_directory_urls' = True
DEBUG   -  Config value 'repo_url' = 'https://gitlab.com/karreg/mkdocs-macros-d2-issue'
DEBUG   -  Config value 'repo_name' = 'mkdocs-macros-d2-issue'
DEBUG   -  Config value 'edit_uri_template' = None
DEBUG   -  Config value 'edit_uri' = 'edit/master/docs/'
DEBUG   -  Config value 'extra_css' = []
DEBUG   -  Config value 'extra_javascript' = []
DEBUG   -  Config value 'extra_templates' = []
DEBUG   -  Config value 'markdown_extensions' = ['toc', 'tables', 'fenced_code']
DEBUG   -  Config value 'mdx_configs' = {}
DEBUG   -  Config value 'strict' = False
DEBUG   -  Config value 'remote_branch' = 'gh-pages'
DEBUG   -  Config value 'remote_name' = 'origin'
DEBUG   -  Config value 'extra' = {}
DEBUG   -  Config value 'plugins' = {'search': <mkdocs.contrib.search.SearchPlugin object at 0xe61d7d9562f0>, 'macros': <mkdocs_macros.plugin.MacrosPlugin
           object at 0xe61d7d956a40>, 'd2': <d2.plugin.Plugin object at 0xe61d7d838eb0>}
DEBUG   -  Config value 'hooks' = {}
DEBUG   -  Config value 'watch' = []
DEBUG   -  Config value 'validation' = {'nav': {'omitted_files': 20, 'not_found': 30, 'absolute_links': 20}, 'links': {'not_found': 30, 'absolute_links': 20,
           'unrecognized_links': 20, 'anchors': 20}}
DEBUG   -  Running `config` event from plugin 'search'
DEBUG   -  Running `config` event from plugin 'macros'
INFO    -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'render_by_default': True, 'force_render_paths': '', 'include_dir': '',
           'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '',
           'j2_comment_start_string': '', 'j2_comment_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
DEBUG   -  [macros] - Project dir '/workspaces/mkdocs-macros-d2-issue'
INFO    -  [macros] - No default module `main` found
DEBUG   -  [macros] - Variables: ['extra', 'config', 'environment', 'plugin', 'git']
INFO    -  [macros] - Extra filters (module): ['pretty', 'relative_url']
DEBUG   -  [macros] - Docs directory: /workspaces/mkdocs-macros-d2-issue/docs
DEBUG   -  [macros] - Includes directory: /workspaces/mkdocs-macros-d2-issue/docs
DEBUG   -  [macros] - Undefined behavior: <class 'jinja2.runtime.DebugUndefined'>
DEBUG   -  Running `config` event from plugin 'd2'
INFO    -  mkdocs-d2-plugin: Using cache at .cache/plugin/d2/db (dbm.ndbm)
DEBUG   -  Running `pre_build` event from plugin 'search'
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /workspaces/mkdocs-macros-d2-issue/site
DEBUG   -  Looking for translations for locale 'en'
DEBUG   -  No translations found here: '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/contrib/search/templates/locales'
DEBUG   -  No translations found here: '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/templates/locales'
DEBUG   -  No translations found here: '/home/gitpod/.local/lib/python3.10/site-packages/mkdocs/themes/mkdocs/locales'
DEBUG   -  Running `files` event from plugin 'd2'
DEBUG   -  Running `nav` event from plugin 'macros'
DEBUG   -  Reading markdown pages.
DEBUG   -  Reading: index.md
DEBUG   -  Running `page_markdown` event from plugin 'macros'
DEBUG   -  [macros] - Page title: Home
DEBUG   -  Copying static assets.
DEBUG   -  Copying media file: 'css/base.css'
DEBUG   -  Copying media file: 'css/bootstrap.min.css'
DEBUG   -  Copying media file: 'css/bootstrap.min.css.map'
DEBUG   -  Copying media file: 'css/brands.min.css'
DEBUG   -  Copying media file: 'css/fontawesome.min.css'
DEBUG   -  Copying media file: 'css/solid.min.css'
DEBUG   -  Copying media file: 'css/v4-font-face.min.css'
DEBUG   -  Copying media file: 'img/favicon.ico'
DEBUG   -  Copying media file: 'img/grid.png'
DEBUG   -  Copying media file: 'js/base.js'
DEBUG   -  Copying media file: 'js/bootstrap.bundle.min.js'
DEBUG   -  Copying media file: 'js/bootstrap.bundle.min.js.map'
DEBUG   -  Copying media file: 'js/darkmode.js'
DEBUG   -  Copying media file: 'search/lunr.js'
DEBUG   -  Copying media file: 'search/main.js'
DEBUG   -  Copying media file: 'search/worker.js'
DEBUG   -  Copying media file: 'webfonts/fa-brands-400.ttf'
DEBUG   -  Copying media file: 'webfonts/fa-brands-400.woff2'
DEBUG   -  Copying media file: 'webfonts/fa-regular-400.ttf'
DEBUG   -  Copying media file: 'webfonts/fa-regular-400.woff2'
DEBUG   -  Copying media file: 'webfonts/fa-solid-900.ttf'
DEBUG   -  Copying media file: 'webfonts/fa-solid-900.woff2'
DEBUG   -  Copying media file: 'webfonts/fa-v4compatibility.ttf'
DEBUG   -  Copying media file: 'webfonts/fa-v4compatibility.woff2'
DEBUG   -  Copying media file: 'assets/stylesheets/mkdocs_d2_plugin.css'
DEBUG   -  Building theme template: sitemap.xml
DEBUG   -  Gzipping template: sitemap.xml
DEBUG   -  Building theme template: 404.html
DEBUG   -  Building markdown pages.
DEBUG   -  Building page index.md
DEBUG   -  Running `page_context` event from plugin 'search'
DEBUG   -  Running `post_build` event from plugin 'search'
DEBUG   -  Running `post_build` event from plugin 'macros'
DEBUG   -  Running `post_build` event from plugin 'd2'
INFO    -  Documentation built in 0.22 seconds

Hope that helps :)

fralau commented 3 days ago

Yes, certainly that gves us something to work with... I still have no idea what's going on, because MkDocs-Macros does not have code to mess up with fenced code (contrary to e.g. MkDocs-Mermaid2).

The issue seems to occur on the pre_build envent.

Do you know so far, which piece of code is saying "Error: The current file is not set for the '!relative' tag. It cannot be used in this context; the intended usage is within markdown_extensions." ? It doesn't look like a like a log entry. Rather, an unhandled exception 🤔

Also, I am not sure what it means exactly, but this might be a pointer: https://github.com/mkdocs/mkdocs/issues/3453 . I am not sure in what file this !relative tag error is occuriring, because it doesn't appear anywhere in your code.

Karreg commented 3 days ago

The error happens here, but I would not be able to say how and why it happens...

That's why the code I gave is minimal. There is nothing but a simple d2 diagram in a single page. I know for sure this is happening at least in this content, though I don't know where exactly in the code it is happening as I am not really knowledgeable in debugging python.

However, your question made me wonder if the issue is really the content. It looks like to me that the issue happens before that.

To make sure of that, I have removed any D2 content from the code I gave you in the opening issue, and the issue still happens.

See here for the empty content.

To summarize: even with no content, only an empty file, the issue happens. So it is unrelated to the content.

Karreg commented 2 days ago

I have executed the build process in debug mode, and here is the instruction that provokes the error: https://github.com/fralau/mkdocs-macros-plugin/blame/3a73707abd48c9df214646d57204cff854bbe98b/mkdocs_macros/plugin.py#L847

The change happened last week.

The debug command is called, but the dump method is executed even if debug mode is disabled. And that is the dump method that itself provokes the error.

Commenting it out prevents the error from happening...

Hope that helps!

fralau commented 2 days ago

Wow. It certainly does!

Now it makes sense. I'll have a look to fix it.

Karreg commented 2 days ago

Just in case if that can help, the stack continues to here: https://github.com/fralau/mkdocs-macros-plugin/blame/3a73707abd48c9df214646d57204cff854bbe98b/mkdocs_macros/util.py#L139

The TypeError happens on a mkdocs.utils.yaml.RelativeDirPlaceholder object

Which down below leads to the initial error... 😃

I am very interested to understand why this happens only when using the d2 plugin... 🤔

fralau commented 2 days ago

It's apparently some object that the object dumper can't process. I should make it more robust, so that it reacts more gracefully.

fralau commented 2 days ago

It was indeed due to the class CustomEncoder, which was susceptible to fail if an object was not printable. I just caught the exception and printed something else.

fralau commented 2 days ago

@Karreg I made a special test case for that, which passed with flying colors.

Could you please test on your side?

Karreg commented 2 days ago

Fix confirmed on both my sample code and a real world documentation 👍 Thanks for the reactivity! It was nice 😃

fralau commented 1 day ago

Thanks! It was a pleasure!