search

JakubAndrysek / MkDoxy

📖 Automatically generates API documentation for your project based on Doxygen comments and code snippets in your markdown files.
https://mkdoxy.kubaandrysek.cz/
MIT License
60 stars 16 forks source link

Custom templates are failing when they contain filter use_code_language. #101

Open jalinei opened 2 months ago

jalinei commented 2 months ago

Hi ! First, thanks for the amazing work, MkDoxy is a banger !

I'm facing the following issue : Overriding memDef.jinja2 lead to the following error : 

  File "/opt/hostedtoolcache/Python/3.12.3/x64/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/mkdocs/__main__.py", line 313, in gh_deploy_command
    build.build(cfg, dirty=not clean)
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/mkdocs/commands/build.py", line 304, in build
    files = config.plugins.on_files(files, config=config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/mkdocs/plugins.py", line 533, in on_files
    return self.run_event('files', files, config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/mkdocs/plugins.py", line 507, in run_event
    result = method(item, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/mkdoxy/plugin.py", line 143, in on_files
    self.generatorBase[project_name] = GeneratorBase(
                                       ^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/mkdoxy/generatorBase.py", line 68, in __init__
    self.templates[name] = Template(fileTemplate)
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/jinja2/environment.py", line 1208, in __new__
    return env.from_string(source, template_class=cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/jinja2/environment.py", line 1105, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
                               ^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
    self.handle_exception(source=source_hint)
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<unknown>", line 13, in template
jinja2.exceptions.TemplateAssertionError: No filter named 'use_code_language'.
Error: Process completed with exit code 1.

I guess somehow the filter is not correctly linked with the custom template ?

Any hints ?

JakubAndrysek commented 2 months ago

Hi a week ago, I have merged commit that fixed rendering code in document section. During this process, PR changed some Jinja files. Please look at this commit and try to change your local files.

https://github.com/JakubAndrysek/MkDoxy/commit/ebfa436cd9973009e722d0330669c98b1c08b097

jalinei commented 2 months ago

But that PR is included in the 1.2.0 release right ? I've checked by downloading the files and I do see the commit #ebfa436

It was actually my first guess : "Am I pulling the last version"

To give more context : I'm only using github CI to deploy direcly, and my CI is getting mkDoxy using pip install mkdoxy

I've checked that pip is linked to 1.2.0.

Either this is really a bug, or it is an edge case due to github CI VM ?

JakubAndrysek commented 2 months ago

Have you found the problem? Does it work fine after changing the custom Jinja file? https://github.com/owntech-foundation/Documentation/commit/dbee6566d3f6dc92e29290241f8c4ace4897253f

JakubAndrysek commented 2 months ago

Might I ask you what you are changing in the templates? Do you have some features that might also be included in the main branch? Thanks

jalinei commented 2 months ago

It works with custom templates that are not using use_code_language filter.

I've tried importing all templates to replace them all -> not working.

I guess the problem is located in generatorBase.py and the filter is not passed correctly to the template. But this problem is quite outside of my dev' skills unfortunately. I'm completely new to templating...

I wanted to tweak memDef.jinja2 for cosmeting changes only. Mainly to had an <hr> tag above each ### titles to have better readability in the class page. ( for instance in this page : https://owntech-foundation.github.io/Documentation/powerAPI/classTwistAPI/ I believe an <hr> tag between functions would improve the experience)

Feature that might be interesting to add in the main branch in the templating : Adding 

----
search: 
  boost: 0.5
----

at the beginning of the file for code.jinja2 and file.jinja2. This would rank down full code snippet in mkdocs search engine. It tends to be the first result each time, while it is not the most interesting bit of documentation.

I've tried pulling that of myself today and yesterday, but without success so far. I've tried escaping using 

{{ raw }}
----
search: 
  boost: 0.5
----
{{ endraw }}

But somehow it is not in the generated md file..

JakubAndrysek commented 2 months ago

I am adding the <hr> tag to the main branch, as you described. It looks nice. image

JakubAndrysek commented 2 months ago

Oh, I have simulated your issue in my dev container, and I am trying to fix it. That is the real bug.

JakubAndrysek commented 2 months ago

The bug has been fixed in the last commit.

JakubAndrysek commented 2 months ago

The boost value can be any positive integer

I am looking at how to set it up correctly.

----
search: 
  boost: 0.5
----
parseInt("0.5") => 0
JakubAndrysek commented 2 months ago

I have tried to go across their documentation, and they support only boosting up. It looks like the default is 1, so it doesn`t exist as a lower value.

If you find something better, I would appreciate it.

jalinei commented 2 months ago

I have tried to go across their documentation, and they support only boosting up. It looks like the default is 1, so it doesn`t exist as a lower value.

If you find something better, I would appreciate it.

https://squidfunk.github.io/mkdocs-material/setup/setting-up-site-search/#search-boosting-rank-down

I've tried to use that example from mkdocs material documentation. Material use the same lunr.js search engine, maybe its a documentation glitch ?

Ranking down commodity pages such as code and files would be more elegant than ranking up all other pages, but if it only takes an integer maybe ranking up the rest would be the way to go ?

jalinei commented 2 months ago

The bug has been fixed in the last commit.

Awesome ! I'll try it today :smile:

JakubAndrysek commented 2 months ago

Oh, I am confused. This project is probably using it, but I did not find any other on the whole GitHub.

So, try it. Just add boost: 0.5 and look at results.

JakubAndrysek commented 2 months ago

I have added new PR with first prototype of implementing the custom configuration exclude-search: files But It needs more time to implement it fully.

jalinei commented 2 months ago

Okay, so here are my findings this morning : With 1.2.1 adding a custom template using use_code_language no longer breaks doc publication. Although project level templating seems not to be working yet.

Here is my CI output : 

Run mkdocs gh-deploy --force
INFO    -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'render_by_default': True, 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO    -  [macros] - Extra variables (config file): ['ownplot', 'social', 'images']
INFO    -  [macros] - Extra filters (module): ['pretty']
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /home/runner/work/Documentation/Documentation/site
INFO    -  Start plugin MkDoxy
INFO    -  -> Start project 'powerAPI'
INFO    -    -> generating Doxygen files
INFO    -  Overwriting template 'memDef' with custom template.
INFO    -  Overwriting template 'code' with custom template.
INFO    -  -> Start project 'controlLibrary'
INFO    -    -> generating Doxygen files
INFO    -  DeprecationWarning: A plugin has set File.page to an instance of Page and it got overwritten. The behavior of this will change in MkDocs 1.6.

Link to my test templates :

Test results :

Maybe something related to #35 ?