Breaks if virtual environment is not activated

[henryiii]

I updated my docs build to the following:

- uv venv
- uv pip install .[docs]
- .venv/bin/python -m mkdocs build --site-dir $READTHEDOCS_OUTPUT/html

And only found out after releasing that pages had been replaced with "rendering errors"! This ruined the permanent release tag docs (https://cibuildwheel.pypa.io/en/v2.18.0/options), and required annoying workarounds to get a stable that had the fix.

This is the error, a file not found error:

INFO     -  [macros] - ERROR # _Macro Rendering Error_

_File_: `options.md`

_FileNotFoundError_: [Errno 2] No such file or directory: 'cibuildwheel'

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/cibuildwheel/checkouts/1821/.venv/lib/python3.12/site-packages/mkdocs_macros/plugin.py", line 527, in render
    return md_template.render(**page_variables)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/cibuildwheel/checkouts/1821/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/home/docs/checkouts/readthedocs.org/user_builds/cibuildwheel/checkouts/1821/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1606, in top-level template code
  File "/home/docs/checkouts/readthedocs.org/user_builds/cibuildwheel/checkouts/1821/docs/main.py", line 15, in subprocess_run
    return subprocess.run(args, check=True, capture_output=True, text=True).stdout
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.12.0/lib/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.12.0/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/docs/.asdf/installs/python/3.12.0/lib/python3.12/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'cibuildwheel'

The fix (besides adding --strict!!!) was to replace .venv/bin/python -m mkdocs (or .venv/bin/mkdocs) with . .venv/bin/activate && mkdocs; for some reason mkdocs-macros breaks down if you don't activate the virtual environment before running it. Python modules are supposed to work without activating the virtual environment.

See https://github.com/pypa/cibuildwheel/pull/1821 & https://github.com/readthedocs/readthedocs.org/pull/11322.

[github-actions[bot]]

Welcome to this project and thank you!' first issue

[fralau]

You're correct: Mkdocs and Mkdocs-Macros do work without virtual environment (that is the way I use them). That depends of course on the absence of incompatible requirements from othermodules.

What you are experiencing appears to be an issue with your configuration.

I see a number of alternative approaches:

1. Update MkDocs and Mkdocs-Macros to their latest versions and see whether that solves the issue.
2. If your setup used to work and stop working, determine what even happened that triggered the problem.
3. Make the simplest copy possible of your project and start adding features of your actual project until it breaks.

Let me know how it goes.

[henryiii]

Ahh, I think I see what is breaking! We have a custom plugin:

def define_env(env: Any) -> None:
    @env.macro
    def subprocess_run(*args: str) -> str:
        return subprocess.run(args, check=True, capture_output=True, text=True).stdout'

That is not picking up the venv's bin folder.

[fralau]

Thanks for the update!

Thanks for the update. My best suggestion for that, is that you create a more specialized macro.

If you use the facilities of Shell, it might raise issues: one of these, is that shell it has own rules for local directories (plus it might open safety issues).

If you write a macro that does exactly what you need, you will have the power of Python, and the rules for the local default directory are more logical (it should be the location of the MkDocs project directory).

Good luck with your project.

[henryiii]

I went with:

def define_env(env: Any) -> None:
    @env.macro
    def subprocess_run(*args: str) -> str:
        env = os.environ.copy()
        scripts = sysconfig.get_path("scripts")
        env["PATH"] = f"{scripts}{os.pathsep}{env.get('PATH', '')}"
        return subprocess.run(args, check=True, capture_output=True, text=True, env=env).stdout

(in https://github.com/pypa/cibuildwheel/pull/1823)

Thanks!

[fralau]

That's also a possible approach!

You might want to use os.path.join() to build your path, but this a secondary consideration.

[henryiii]

You are thinking of os.sep/os.path.sep (/ or \), this is os.pathsep/os.path.pathsep (; or :). :)

[fralau]

Oh, I see... That's another kettle of fish. 🙂