jinja2.exceptions.TemplateNotFound: toggle-primary-sidebar.html

[AlexanderJuestel]

Describe the bug

context When I creating the the documentation page of GemGIS (https://gemgis.readthedocs.io/en/latest/), I have recently started to see an error: jinja2.exceptions.TemplateNotFound: toggle-primary-sidebar.html.

expectation No error to occur

bug The creating of the docs fails --> https://readthedocs.org/projects/gemgis/builds/20023049/

Reproduce the bug

The creating of the docs fails --> https://readthedocs.org/projects/gemgis/builds/20023049/

List your environment

No response

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[jayaddison]

Hi @AlexanderJuestel - the earliest failed build that I see with this error is from 2023-03-02: https://readthedocs.org/projects/gemgis/builds/19649141/

That looks like it's fairly soon after v1.0.0 of sphinx-book-theme was released (including some breaking changes, most likely).

Have you consulted the changelog for any relevant items?

[AlexanderJuestel]

@jayaddison Thanks for the hint! I will check out the changelog. A temporary solution could be to pin the book theme to an earlier version?

[jayaddison]

@AlexanderJuestel it could be, yep - if possible though, I'd recommend trying to find the relevant change that introduced the problem, and whether it's a bug/known-issue. If so, then it'll be easier to follow along and unpin the version (or to change this theme appropriately when ready to upgrade).

[AlexanderJuestel]

@jayaddison One thing that I could find right away is https://github.com/executablebooks/sphinx-book-theme/pull/677/commits/60bcb7df8ea7536d09495223a32261f0dcfd0052#diff-d9e027334fa1c4b5093d44ff026f62f9e4d092fa735da8ffd1e6b96c0f645f3f

However, the changes made to https://github.com/sphinx-doc/sphinx/commits/master/sphinx/jinja2glue.py were not very revealing to me about what could have been broken.

[jayaddison]

Ok, thanks @AlexanderJuestel. The jinja2glue.py file seems to iterate through a series of template loaders (each with their own paths configured), and one of those should find and load the toggle-primary-sidebar.html file (that exists inside a components directory).

Currently I'm looking into the code here to try to figure out whether this additional file path is being considered at load-time: https://github.com/executablebooks/sphinx-book-theme/blob/c2664ddfbe2c3fcfaa819a57835f00ca58346cb9/src/sphinx_book_theme/__init__.py#L191-L198

[AlexanderJuestel]

@jayaddison thanks for your efforts! I read somewhere else that it may be related to templates. Keep me updated if you find something.

[jayaddison]

A self-correction: I now believe that this is the relevant code to look at: https://github.com/executablebooks/sphinx-book-theme/blob/c2664ddfbe2c3fcfaa819a57835f00ca58346cb9/src/sphinx_book_theme/__init__.py#L259-L261

Perhaps that additional entry in the template path list isn't being retained, for some reason.

[jayaddison]

Some more notes:

Dependency versions

• Downgrading sphinx-book-theme to v0.3.3 does allow gemgis documentation to build
• Building with v1.0.0 fails due to an unrelated error
• Building with v1.0.1 fails due to the error reported here

So there does appear to be an incompatibility related to v1.x that gemgis may need to address (or that may be a bug here).

Template origin

The sphinx-book-theme Sphinx theme depends upon (but does not inherit from?) pydata-sphinx-theme.

The toggle-primary-sidebar.html file is part of sphinx-book-theme.

But the error that appears in the build output seems to be in the context of pydata-sphinx-theme:

Extension error (pydata_sphinx_theme):
Handler <function update_and_remove_templates at 0x...> for event 'html-page-context' threw an exception (exception: toggle-primary-sidebar.html)

Why is that? Could there be a mismatch between the template search paths for the two themes?

[jayaddison]

The sphinx-book-theme Sphinx theme depends upon (but does not inherit from?) pydata-sphinx-theme.

Ok, sphinx-book-theme does in fact inherit from pydata-sphinx-theme:

https://github.com/executablebooks/sphinx-book-theme/blob/c2664ddfbe2c3fcfaa819a57835f00ca58346cb9/src/sphinx_book_theme/theme/sphinx_book_theme/theme.conf#L2

[jayaddison]

Sorry, @AlexanderJuestel - I've taken more of a look at this, but haven't been able to figure out what the root of the problem is. As you suggest, limiting the sphinx-book-theme version (to <1.0.0, or ==0.3.3, for example) should provide a workaround.

[AlexanderJuestel]

@jayaddison thanks for checking, I will try limiting the version and get back to you ASAP

[AlexanderJuestel]

@jayaddison Limiting the version to 0.3.3 resolves the issue. Closing this for now.

[adam-grant-hendry]

The toggle-primary-sidebar.html file is part of sphinx-book-theme.

But the error that appears in the build output seems to be in the context of pydata-sphinx-theme:

Extension error (pydata_sphinx_theme): Handler <function update_and_remove_templates at 0x...> for event 'html-page-context' threw an exception (exception: toggle-primary-sidebar.html) Why is that? Could there be a mismatch between the template search paths for the two themes?

Yes; pydata-sphinx-theme manually adds their components folder to the config.templates_path in their __init__.py/setup() method:

app.config.templates_path.append(str(theme_path / "components"))

which alters the default search path order for templates, calculated in sphinx/jinja2glue.py/BuiltinTemplateLoader.init().

Per the sphinx docs, the default search order is:

• user's template_path
• selected theme
• base theme (i.e. what is specified in theme.conf by inherit =)

However, debugging sphinx-build with the sphinx-book-theme, we see pathchain is initially set properly by sphinx/theming.py/get_theme_dirs():

but is messed up when it gets to the point of prepending explicit template paths:

NOTE: I debug sphinx-build in VSCode on Windows 10 using the following launch.json configuration:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Sphinx Debugger",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/.venv/Scripts/sphinx-build.exe",
            "args": ["-b", "html", "src", "build"],
            "console": "integratedTerminal",
            "cwd": "${workspaceFolder}/docs",
            "justMyCode": false,
        }
    ]
}

and then running from the Run and Debug view:

and have the following package versions:

python: 3.8.10 Sphinx: 6.2.1 jinja2: 3.1.2 sphinx-book-theme: 1.0.1 pydata-sphinx-theme: 0.14.1

[adam-grant-hendry]

but is messed up when it gets to the point of prepending explicit template paths:

Actually, it appears sphinx-book-theme also appends to the templates_path, but what is odd is that the "components" portion of the path is missing.

[adam-grant-hendry]

Just wanted to capture what's going on during a build, specifically this is run:

sphinx-build -b html src build

In this case, conf.py has sphinx_book_theme in extensions, but not pydata_sphinx_theme. i.e.:

#conf.py

extensions - [
    'sphinx_book_theme',
    # not 'pydata_sphinx_theme',
    ...
]

Starting from the sphinx-build entry point (i.e. sphinx.cmd.build:main; see pyproject.toml), the following happens in sphinx/application.py/Sphinx.__init__():

1. The conf.py file is read
2. The sphinx_book_theme setup() method is run, after which templates_path is:

['project\\.venv\\Lib\\site-packages\\sphinx_book_theme\\theme\\sphinx_book_theme\\components']

3. config.init_values() is run, which overrides templates_path to:

['_templates']

4. _init_builder() is run, calling sphinx/builds/html/__init__.py/StandaloneHTMLBuilder.init(), which calls StandaloneHTMLBuilder.init_templates().

5. An HTMLThemeFactory creates 'sphinx_book_theme', which reloads pydata_sphinx_theme by calling its setup(), after which templates_path is:

['_templates', 'C:\\Users\\hendra11\\Code\\external\\poetry_plugin_constrain\\.venv\\Lib\\site-packages\\pydata_sphinx_theme\\theme\\pydata_sphinx_theme\\components']

The sphinx_book_theme setup() is never re-run, so sphinx is unaware of the sphinx_book_theme\\components path, so sphinx cannot find toggle-primary-sidebar.html.

[adam-grant-hendry]

It looks like this issue was fixed in sphinx-book-theme PR #566 precisely how I would have though to fix it: create a config-inited event:

app.connect("config-inited", update_general_config)

but it looks like it was changed to a 'builder-inited' event in PR #691:

app.connect("builder-inited", update_general_config)

This looks like it was a mistake.

[adam-grant-hendry]

I can confirm reverting to:

app.connect("config-inited", update_general_config)

and

def update_general_config(app, config):  # add config argument back
    ...

fixes the problem.

[adam-grant-hendry]

@AlexanderJuestel @jayaddison You have sphinx-book-theme added to extensions when it is only intended to be added to html_theme. Remove sphinx-book-theme from the extensions list, and your problem should be fixed.