Documentation generation failure when referencing a class not declared yet in type annotation

dureuill commented 3 years ago

Thank you for maintaining pdoc! I encountered an issue when running the tool on code that can be minimized to the following:

class Foo:
    def bar(self) -> Bar:
        return Bar()

class Bar:
    def foo(self) -> Foo:
        return Foo()

Command line used:

pdoc --html /tmp/testdoc.py

Expected Behavior

I would expect the tool to generate documentation on the example above, with the following output:

expected

At first I doubted that this kind of recursive annotations are actually allowed (I'm pretty new to annotation typing in python3), but it looks like mypy accepts them fine:

$ mypy /tmp/testdoc.py 
Success: no issues found in 1 source file

Actual Behavior

$ pdoc --html /tmp/testdoc.py 
Traceback (most recent call last):
  File "/home/tetrane/.local/lib/python3.7/site-packages/pdoc/__init__.py", line 225, in import_module
    module = importlib.import_module(module_path)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/testdoc.py", line 1, in <module>
    class Foo:
  File "/tmp/testdoc.py", line 2, in Foo
    def bar(self) -> Bar:
NameError: name 'Bar' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tetrane/.local/bin/pdoc", line 10, in <module>
    sys.exit(main())
  File "/home/tetrane/.local/lib/python3.7/site-packages/pdoc/cli.py", line 534, in main
    for module in args.modules]
  File "/home/tetrane/.local/lib/python3.7/site-packages/pdoc/cli.py", line 534, in <listcomp>
    for module in args.modules]
  File "/home/tetrane/.local/lib/python3.7/site-packages/pdoc/__init__.py", line 609, in __init__
    module = import_module(module)
  File "/home/tetrane/.local/lib/python3.7/site-packages/pdoc/__init__.py", line 228, in import_module
    .format(module, e.__class__.__name__, e))
ImportError: Error importing '/tmp/testdoc.py': NameError: name 'Bar' is not defined

Steps to Reproduce

Copy the python script above in a script called /tmp/testdoc.py
Run the command: pdoc --html /tmp/testdoc.py

Additional info

pdoc version: 0.9.2
python version: 3.7.3
mypy version: 0.812
There appears to be a workaround to this issue:

class Bar:
    pass

class Foo:
    def bar(self) -> Bar:
        return Bar()

class Bar:
    def foo(self) -> Foo:
        return Foo()

however, using it makes it so mypy now fails on the code:

$ mypy /tmp/testdoc.py 
/tmp/testdoc.py:8: error: Name 'Bar' already defined on line 1
Found 1 error in 1 file (checked 1 source file)

As a workaround, maybe an unknown type in an annotation could be promoted to a warning rather than an error? This would not necessilarily fix the underlying issue but would permit the documentation generation in this case.
Apologies if this issue is a duplicate. I had a look at the issue tracker but could only find #245 which seems related in behavior, but has a different cause (TYPE_CHECKING runtime value?)
Thank you for reading!

kernc commented 3 years ago

python version: 3.7.3

In Python <=3.7, typing annotations are evaluated at definition time. In Python 3.8+, they are evaluated only as needed by type checking utilities. Thus, the error NameError: name 'Bar' is not defined makes sense in your case.

Your options are:

make the annotation a forward reference (expression wrapped in string), e.g.:
```
  def bar(self) -> 'Bar': ...
```
enable the future behavior by importing at the top of the file:
```
from __future__ import annotations
```

dureuill commented 3 years ago

Thank you for the answer, looks like this is due to my version of Python then!

I was misled by mypy actually having a different behavior in this case, apparently.

I will use one of the solutions you proposed and close this issue. Thank you!

kernc commented 3 years ago

Mypy works because it does static code analysis on raw text files, sans execution.

pdoc3 / pdoc