search

NiklasRosenstein / pydoc-markdown

Create Python API documentation in Markdown format.
http://niklasrosenstein.github.io/pydoc-markdown/
Other
458 stars 103 forks source link

Crash with databind 4.5.0 #323

Closed janbuchar closed 6 months ago

janbuchar commented 6 months ago

Environment

Describe the bug

After the release of databind 4.5.0, it seems that our docs pipeline fails because it can't find databind.json (which was removed, apparently?) - please see https://github.com/apify/apify-sdk-python/actions/runs/8375038166/job/22931545213#step:8:70

Reverting databind to 4.4.2 seems to have fixed the issue temporarily.

Expected behavior

No crashes.

NiklasRosenstein commented 6 months ago

See #322 -- I can't explain why multiple people hit this issue, when it appeared to be a flake in the other PR and installing databind.json == 4.5.0 really should just give you databind == 4.5.0. Does your CI run with caches and if yes could you try clearing those?

I have yet to find a reliable reproduction of the issue.

NiklasRosenstein commented 6 months ago

Actually I think I just found an issue, but not quite the one your CI was having.

#!/bin/bash
(
set -e
tmp=$(mktemp -d)
trap 'rm -rf $tmp' EXIT
cd $tmp
python3 -m venv venv
source venv/bin/activate
pip install pydoc-markdown==4.8.2 databind.json==4.4.2
pydoc-markdown --version
pip install databind.json==4.5.0
pydoc-markdown --version  # ModuleNotFoundError: No module named 'databind.json'
)

And I think this is because Pip removes the files that are no longer installed by databind.json==4.5.0 after it installed the new version of databind==4.5.0. I.e. the files effectively changed ownership, but Pip believes that they need to be removed for the upgrade from databind.json 4.4.2 -> 4.5.0.

I don't see a way around this without either reverting the Databind change or fixing this behaviour in Pip; but for what it's worth this shouldn't happen on a fresh install.

#!/bin/bash
(
set -e
tmp=$(mktemp -d)
trap 'rm -rf $tmp' EXIT
cd $tmp
python3 -m venv venv
source venv/bin/activate
pip install pydoc-markdown==4.8.2 databind.json==4.5.0
pydoc-markdown --version
)
NiklasRosenstein commented 6 months ago

This looks like exactly that issue; only in reverse (databind merges the packages instead of splitting them).

https://github.com/pypa/pip/issues/8509

janbuchar commented 6 months ago

Does your CI run with caches and if yes could you try clearing those?

None that I'm aware of. At least the "Install Python dependencies" step does not use any, and the one for node.js setup should not influence the build.

I have yet to find a reliable reproduction of the issue.

Unfortunately, this fails rather reliably in our project.

for what it's worth this shouldn't happen on a fresh install.

What do you mean by fresh install, a new venv? I tried that and still get the error. If you want to reproduce it, use tag v1.7.0, for example - we downgraded pydoc-markdown in master.

NiklasRosenstein commented 6 months ago

Not sure what I did different last time, but I can now reproduce it.

Canonical issue for tracking: https://github.com/NiklasRosenstein/pytho##n-databind/issues/64

NiklasRosenstein commented 6 months ago

This should be fixed with databind 4.5.1

https://github.com/NiklasRosenstein/python-databind/issues/64#issuecomment-2031590140