Fails on __init__() method with valid but peculiar indentation

[dirkroorda]

If the __init__ method of a class has code that extrudes to the left margin an error is produced.

Here is a minimal example

class Aaa:
    # def method(self):
    def __init__(self):
        print(
            """
aaa
"""
        )

If we rename the __init__ method to anything else, e.g. method, there is no problem. The expected behaviour is that there is no problem at all.

Here is the complete set up:

docs/aaa.md
aaa.py
mkdocs.yml

with this content

aaa.md

::: aaa

mkdocs.yml

site_name: bug

theme:
  name: material

plugins:
  - mkdocstrings

The command is

mkdocs -v build

and the output is

DEBUG    -  Loading configuration file: ~/local/mkdocstrings/mkdocs.yml
DEBUG    -  Loaded theme configuration for 'material' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/material/mkdocs_theme.yml': {'language': 'en', 'direction':
            None, 'features': [], 'palette': {'primary': None, 'accent': None}, 'font': {'text': 'Roboto', 'code': 'Roboto Mono'}, 'icon': None, 'favicon': 'assets/images/favicon.png',
            'include_search_page': False, 'search_index_only': True, 'static_templates': ['404.html']}
DEBUG    -  Config value: 'config_file_path' = '~/local/mkdocstrings/mkdocs.yml'
DEBUG    -  Config value: 'site_name' = 'bug'
DEBUG    -  Config value: 'nav' = None
DEBUG    -  Config value: 'pages' = None
DEBUG    -  Config value: 'site_url' = ''
DEBUG    -  Config value: 'site_description' = None
DEBUG    -  Config value: 'site_author' = None
DEBUG    -  Config value: 'theme' = Theme(name='material', dirs=['/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/material',
            '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mkdocs/templates'], static_templates=['sitemap.xml', '404.html'], locale=Locale('en'), language='en',
            direction=None, features=[], palette={'primary': None, 'accent': None}, font={'text': 'Roboto', 'code': 'Roboto Mono'}, icon=None, favicon='assets/images/favicon.png',
            include_search_page=False, search_index_only=True)
DEBUG    -  Config value: 'docs_dir' = '~/local/mkdocstrings/docs'
DEBUG    -  Config value: 'site_dir' = '~/local/mkdocstrings/site'
DEBUG    -  Config value: 'copyright' = None
DEBUG    -  Config value: 'google_analytics' = None
DEBUG    -  Config value: 'dev_addr' = Address(host='127.0.0.1', port=8000)
DEBUG    -  Config value: 'use_directory_urls' = True
DEBUG    -  Config value: 'repo_url' = ''
DEBUG    -  Config value: 'repo_name' = ''
DEBUG    -  Config value: 'edit_uri' = ''
DEBUG    -  Config value: 'extra_css' = []
DEBUG    -  Config value: 'extra_javascript' = []
DEBUG    -  Config value: 'extra_templates' = []
DEBUG    -  Config value: 'markdown_extensions' = ['toc', 'tables', 'fenced_code']
DEBUG    -  Config value: 'mdx_configs' = {}
DEBUG    -  Config value: 'strict' = False
DEBUG    -  Config value: 'remote_branch' = 'gh-pages'
DEBUG    -  Config value: 'remote_name' = 'origin'
DEBUG    -  Config value: 'extra' = {}
DEBUG    -  Config value: 'plugins' = PluginCollection([('mkdocstrings', <mkdocstrings.plugin.MkdocstringsPlugin object at 0x7f9bc85e18b0>)])
DEBUG    -  mkdocstrings.plugin: Adding extension to the list
DEBUG    -  mkdocstrings.plugin: Added a subdued autorefs instance <mkdocs_autorefs.plugin.AutorefsPlugin object at 0x7f9bc86c4be0>
DEBUG    -  mkdocs_autorefs.plugin: Adding AutorefsExtension to the list
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /Users/dirk/local/mkdocstrings/site
DEBUG    -  Looking for translations for locale 'en'
DEBUG    -  No translations found here: '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mkdocs/templates/locales'
DEBUG    -  No translations found here: '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/material/locales'
DEBUG    -  Reading markdown pages.
DEBUG    -  Reading: aaa.md
DEBUG    -  mkdocstrings.extension: Matched '::: aaa'
DEBUG    -  mkdocstrings.extension: Using handler 'python'
DEBUG    -  mkdocstrings.handlers.python: Opening 'pytkdocs' subprocess
DEBUG    -  mkdocstrings.extension: Collecting data
DEBUG    -  mkdocstrings.handlers.python: Preparing input
DEBUG    -  mkdocstrings.handlers.python: Writing to process' stdin
DEBUG    -  mkdocstrings.handlers.python: Reading process' stdout
DEBUG    -  mkdocstrings.handlers.python: Loading JSON output as Python object
ERROR    -  mkdocstrings.extension: unexpected indent (<unknown>, line 1)
            Traceback (most recent call last):
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/cli.py", line 205, in main
                output = json.dumps(process_json(line))
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/cli.py", line 114, in process_json
                return process_config(json.loads(json_input))
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/cli.py", line 91, in process_config
                obj = loader.get_object_documentation(path, members)
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/loader.py", line 358, in get_object_documentation
                root_object = self.get_module_documentation(leaf, members)
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/loader.py", line 426, in get_module_documentation
                root_object.add_child(self.get_class_documentation(child_node))
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/loader.py", line 474, in get_class_documentation
                attributes_data.update(get_instance_attributes(class_.__init__))
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/parsers/attributes.py", line 134, in get_instance_attributes
                nodes = get_nodes(func)
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pytkdocs/parsers/attributes.py", line 22, in get_nodes
                return ast.parse(dedent(source)).body
              File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ast.py", line 50, in parse
                return compile(source, filename, mode, flags,
              File "<unknown>", line 1
                def __init__(self):
            IndentationError: unexpected indent
ERROR    -  Error reading page 'aaa.md':
ERROR    -  Could not collect 'aaa'

Aborted with a BuildError!

It seems that pytkdocs does work but produces output on which mkdocstrings crashes. Whether this is a bug in mkdocstrings or in pytkdocs is beyond me.

System:

• pytkdocs version 0.12.0
• Python version: 3.9
• OS: Macos

[pawamoy]

Hello @dirkroorda, thank you for the report!

It seems that pytkdocs does work but produces output on which mkdocstrings crashes. Whether this is a bug in mkdocstrings or in pytkdocs is beyond me.

Actually it seems that compile itself (which is called by ast.parse) has trouble handling such code separately. Maybe the dedent is botching the source but I doubt that? I'll run some tests.

[pawamoy]

In the meantime you could try to dedent the string yourself as a workaround:

from textwrap import dedent

class Aaa:
    # def method(self):
    def __init__(self):
        print(
            dedent(
                """
                aaa
                """
            )
        )

inspect.cleandoc might handle more cases:

from inspect import cleandoc

class Aaa:
    # def method(self):
    def __init__(self):
        print(
            cleandoc(
                """
                aaa
                """
            )
        )

[dirkroorda]

Yes, and another work around is to put such code in an other method and call that method from init.

The puzzling fact is that such code does not lead to problems in non-init methods, so I wonder if mkdocstrings has a separate pass over the code before feeding it to the parser, or does it tweak the output of the parser in some way?

[pawamoy]

__init__ methods are in fact the only methods that pytkdocs actually parses in search of annotations/docstrings for instances attributes. That explains why it does not fail the same way on other methods with the same code.

[pawamoy]

I suspect compile is not happy to see code starting with indentation (relative to the multiline string!), and am not sure how to make it happy with it :/

[pawamoy]

Yup I think it won't be easy to fix this, unfortunately. To parse this code, compile needs the code before it, leading to the indentation it has.

Note however that I'm working on a replacement of pytkdocs that will not suffer from this issue :slightly_smiling_face:

Closing as won't fix, but feel free to further comment on this if you think this should really be fixed.

[dirkroorda]

Gladly applied the dedent workaround everywhere. Did not know it before, makes my code a bit nicer.