feature: Docstring pre-formater

[nbanyan]

Scenario: Because the docstrings for SaltStack modules are viewed by users through a CLI, I need the look and format of the docstrings I write to match the rest of Salt. However, the style used is neither Google nor Sphinx but uses elements of both causing the mkdocstrings output to be messy. So I'm looking for ways to alter the docstrings to Google (or Sphinx) style before Griffe parses them, but without changing the script files.

Alternatives: Extensions looked promising, but all their event hooks happen after docstrings are parsed. Post processing this way might work, but I don't now yet if changing things (like argument defaults, adding sections, etc.) or triggering a re-parsing is feasible. Writing an MkDocs wrapper extension for mkdocstrings might be possible, but I expect it'd be messy.

Feature: Would it be feasible to create a pre-processor option similar to the Extensions? Just a single event hook for after a module file is loaded, but before it is parsed, to allow a script to walk through the docstrings and make corrections. If this is possible, this would be the cleanest option by far.

[pawamoy]

Hi @nbanyan, thanks for the request.

Extensions looked promising, but all their event hooks happen after docstrings are parsed.

That's not correct. Griffe actually never parses docstrings by itself. Parsing is only triggered once objects are being rendered by mkdocstrings-python. So it means that extensions always run before any docstring parsing has been done (unless such an extension has already manually triggered parsing). Could you point at the docs/code that lead you to think otherwise, so we can maybe reword it?

So to pre-process your docstrings, you can do something like this:

import griffe

class PreprocessSaltStackDocstrings(griffe.Extension):
    def on_instance(self, *, obj: griffe.Object, **kwargs) -> None:
        if obj.docstring:
            _preprocess(obj.docstring)
            # ...where _preprocess is a custom function of yours,
            # and modifies the docstring.value attribute.

[nbanyan]

Thank you. That does work well. All the Extension methods' arguments are objects that already have many different members (annotations, classes, etc.) which strongly gives the impression of them being the result of the source having already been processed.

I found that the mkdocstrings rendering seems to only use the docstring member for the body, so replacing docstring with a new Docstring works well.

Originally, I was testing with on_package_loaded since that's what griffe_modernized_annotations uses, but your suggestion of on_instance is better since it happens earlier and removes the need to walk through the module manually.

def on_instance(self, *, obj: griffe.Object, **kwargs) -> None:
    if (obj.has_docstring):
        docstring = obj.docstring
        source = _modify_docstring(obj.source, docstring.parser)
        obj.docstring = griffe.Docstring(source,
                                         lineno=docstring.lineno,
                                         endlineno=docstring.endlineno,
                                         parent=docstring.parent,
                                         parser=docstring.parser,
                                         parser_options=docstring.parser_options)

[pawamoy]

Great!

All the Extension methods' arguments are objects that already have many different members (annotations, classes, etc.) which strongly gives the impression of them being the result of the source having already been processed.

Oh I see, that makes sense. The source has been processed indeed, but docstrings haven't been parsed as Google, Numpydoc or whatever style :slightly_smiling_face:

I found that the mkdocstrings rendering seems to only use the docstring member for the body, so replacing docstring with a new Docstring works well.

If your new docstring only differs by its value attribute, it could be easier to just re-assign it.

---

I believe I can close this then, thanks for the update!