ImportError: cannot import name 'DocstringAttr' from 'docstring_parser.common'

[magnussommarsjo]

From time to time I run into the following issue

$ pydoclint .
Traceback (most recent call last):
  File "/home/mlengqui/miniconda3/envs/sohvida_dc_update/bin/pydoclint", line 5, in <module>
    from pydoclint.main import main
  File "/home/mlengqui/miniconda3/envs/sohvida_dc_update/lib/python3.9/site-packages/pydoclint/main.py", line 18, in <module>
    from pydoclint.visitor import Visitor
  File "/home/mlengqui/miniconda3/envs/sohvida_dc_update/lib/python3.9/site-packages/pydoclint/visitor.py", line 5, in <module>
    from pydoclint.utils.arg import Arg, ArgList
  File "/home/mlengqui/miniconda3/envs/sohvida_dc_update/lib/python3.9/site-packages/pydoclint/utils/arg.py", line 4, in <module>
    from docstring_parser.common import DocstringAttr, DocstringParam
ImportError: cannot import name 'DocstringAttr' from 'docstring_parser.common' (/home/mlengqui/miniconda3/envs/sohvida_dc_update/lib/python3.9/site-packages/docstring_parser/common.py)

This is even though I have the docstring_parser_fork installed.

$ pip list | grep docstring_parser
docstring_parser           0.16
docstring_parser_fork      0.0.9

Is this due to that pydoclint is using the 'regular' version instead of the fork in this case?

[jsh9]

Hi @magnussommarsjo , maybe it's best to uninstall docstring_parser to avoid naming conflicts. I added DocstringAttr in the fork, but apparently your Python was still looking into the original "docstring_parser" library.

[magnussommarsjo]

Removing by simply pip uninstall docstring-parser will then give an ModuleNotFoundError: No module named 'docstring_parser.common' when running pydoclint ..

Why I ending up with both is that I have another dependency on docstring-parser called kfp. Below is the "inverted" dependency graph.

docstring-parser v0.16
└── kfp v1.8.22 (*)
docstring-parser-fork v0.0.9
└── pydoclint v0.5.8
    ├── my-project

[magnussommarsjo]

Here is a minimal code sample that will recreate this issue using conda.

conda create -n testenv python=3.9 -y
conda activate testenv
pip install kfp pydoclint
pydoclint .

[jsh9]

Could you run pydoclint in a separate environment?

My earlier suggestion to uninstall docstring_parser wouldn't work, because it turned out that both docstring_parser and docstring_parser_fork were installed in the same folder: /Users/...../anaconda3/envs/testenv/lib/python3.9/site-packages/docstring_parser/*, because they share the same module name, which is docstring_parser.

There is unfortunately not a perfect fix for this situation. I prefer not to change the module name of the forked library (otherwise keeping up with the latest changes on the original repo would be difficult). And in the original repo, I reported a bug (Issue #81 there) but there was no response (which is why I had to fork it).

[magnussommarsjo]

Well I can run it in another environment. But that is not a great workflow for me or my team. This would basically mean that as soon as I need to commit and run "pre-commit" where my linting checks are run. I'll then need to temporarily switch environment and then back again when I need to run testing with pytest.

I understand that this issue might be hard to fix. But anyhow I wanted to raise it so that you are aware of it. 😉

[jsh9]

May I see your tox or pre-commit config? I've always been running pydoclint in a dedicated env in tox, so I never ran into this issue before.

And btw I just found a workaround:

conda create -n testenv python=3.9 -y
conda activate testenv
pip install kfp
pip install pydoclint
pydoclint .

The key is to split up kfp and pydoclint into 2 commands, and make sure you install pydoclint last so that the old (original) docstring_parser gets overwritten by docstring_parser_fork. You can inspect the file .../anaconda3/envs/testenv/lib/python3.9/site-packages/docstring_parser/common.py to double check: the forked version has 298 lines while the original version has 228 lines.