Maintenance: local documentation raises exception when run in docker

[dreamorosi]

Summary

When running the documentation locally using Docker the mkdocs serve command seems to fail

Running docker run --rm -it -p 8000:8000 -v ${PWD}:/docs powertools-typescript/docs, causes this exception:

Traceback (most recent call last):
  File "/usr/local/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/__main__.py", line 268, in serve_command
    serve.serve(**kwargs)
  File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 54, in serve
    config = get_config()
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 43, in get_config
    config = load_config(
             ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/config/base.py", line 374, in load_config
    errors, warnings = cfg.validate()
                       ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/config/base.py", line 231, in validate
    run_failed, run_warnings = self._validate()
                               ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/config/base.py", line 188, in _validate
    self[key] = config_option.validate(value)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/config/base.py", line 55, in validate
    return self.run_validation(value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/config/config_options.py", line 868, in run_validation
    return theme.Theme(**theme_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/theme.py", line 61, in __init__
    self._load_theme_config(name)
  File "/usr/local/lib/python3.11/site-packages/mkdocs/theme.py", line 126, in _load_theme_config
    theme_dir = utils.get_theme_dir(name)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/mkdocs/utils/__init__.py", line 259, in get_theme_dir
    return os.path.dirname(os.path.abspath(theme.load().__file__))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 399, in abspath
TypeError: expected str, bytes or os.PathLike object, not NoneType
FATA[0000] exit status 1

The issue seem to have been introduced in this PR and only occurs when using Docker. If I run the docs using Python directly on my machine I am able to run it normally.

Likewise, the build for the docs is also unaffected.

We should investigate why this is happening.

Why is this needed?

So that contributors in this repo can run the docs locally without having to install Python on their machines.

Which area does this relate to?

Other

Solution

No response

Acknowledgment

• [X] This request meets Powertools for AWS Lambda (TypeScript) Tenets
• [ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[am29d]

Bumping docker image to 2a703999163cdb8257a85849fb2d39914a82587769f95c297bb3f01acf72a609 which is 9.5.35 fixes the issue.

[github-actions[bot]]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[leandrodamascena]

Hey @am29d and @dreamorosi! dedicated some time tonight to check this issue, and I believe I've identified the root cause. Before going into the details, I must say that I don't understand why we need to use pip-compile to generate a requirements.txt file with hashes and transitive dependencies in this situation. I mean, I understand that this approach can sometimes be useful to prevent conflicts, but in our current situation it seems to be having the opposite effect: it's actually creating conflicts. Maybe in the past it was useful, but right now I don't see any need to use this.

So, can you try to reproduce the fix in your local environment? If it works, I can submit a PR:

1 - Remove the current docker image: docker image rm powertools-typescript/docs --force 2 - Clear the contents of the docs/requirements.txt file. 3 - Copy the content from docs/requirements.in to docs/requirements.txt 4 - Remove the flag --require-hashes from the docs/Dockerfile (line 5) 5 - Build the documentation: docker build -t powertools-typescript/docs ./docs/ 6 - Run the documentation: docker run --rm -it -p 8000:8000 -v ${PWD}:/docs powertools-typescript/docs

I checked the pipeline for building the documentation and it should not be affected when using mkdocs build to build the documentation directly.

While we're at it, why don't you also use the Makefile to automate some things? This isn't exclusive to npm scripts and you can maintain them, but I think there's a universe of people who might take a while to figure out which npm script to run, whereas the Makefile is more straightforward.

Thanks

[dreamorosi]

Hi Leo, thanks for looking into this, this helps.

I have tested the steps you shared and confirmed that it works, however as discussed offline we might not be able to implement them. Removing the lock file would affect our supply chain posture, and also our OpenSSF scorecard.

Ideally, if there's a conflict between the base image we are using and the additional dependencies we are installing, we should try to fix that instead.

I don't have a good solution yet, but we can think of something.

Regarding the Makefile suggestion, I think for now we'll stick to npm. I don't have anything against using makefiles specifically but at least from my personal experience I don't think it's a standard that is very common in the Node.js ecosystem.

[leandrodamascena]

Hi @dreamorosi, thanks for sharing the OpenSSF scorecard check with me. There is actually a check for pinned dependencies if a pip install command is found.

The same issue is happening with Python and I figured out the problem. The dependabot is bumping packages and dependencies in the requirements.txt file, but it is not smart enough - don't ask me why - to detect that it should bump the transitive dependency mkdocs to 1.6.1, which is where the conflict is happening.

If you look here, you will see that it is using mkdocs 1.6.0, even though the latest is 1.6.1.

I created a new requirements.txt file with the dependencies updated and it's working now.

This is the first time this has happened and I don't know if it will happen again in the future, but it seems like we don't have many options now.

I'm sending a PR to fix this.

[github-actions[bot]]

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.