Modifications for better results on Django projects?

[odigity]

Problem Description

The default behavior of pdoc, when applied to a Django project, leaves much to be desired, in my subjective opinion. Because it's hard to explain in an issue post, I've created a mini-project with README.md to record my thoughts and findings:

https://github.com/odigity/pdocdemo/tree/main/testcases/djangodoc

Proposal

I believe it would be beneficial to create some kind of extension to customize pdoc for Django projects. Unfortunately, I'm new to both Django and pdoc, and not yet an expert at even Python, so I don't have a strong sense of how to best implement such thing.

[mhils]

Thanks for the super detailed description again! 🍰

I believe it would be beneficial to create some kind of extension to customize pdoc for Django projects.

Let me preface this by saying that pdoc targets library documentation and not really Django apps if that makes sense. I would say this isn't really pdoc's main use case (which I would like to keep focused on), but of course it's nice if pdoc is useful for you and others on Django codebases as well. I absolutely would not mind seeing a pdoc-django package on PyPI that internally uses pdoc with some Django-specific optimizations. I'd be happy to promote that on the pdoc website, but I don't want to build or maintain it. :)

Coming to some of the proposals in your repo:

In other words, I'm loading each module, converting it to a pdoc.doc.Module object, and using the .docstring and .members properties to determine if the module is worth including.

I think this is a reasonable proposal, but somewhat tricky to implement as we don't know if any members will be public. A simpler fix for this would be to implement negated module specs (https://github.com/mitmproxy/pdoc/issues/310#issuecomment-954019271) and just have an example with default excludes for Django.

Alternatively we change pdoc.pdoc to also accept pdoc.doc.Module objects as part of the modules argument. Then you can do your custom module filtering in your own code before handing off the module list to pdoc. That would also be a generally useful change I would like to see.

Module Contents flooded with noise from attributes automatically added by Django.

Yes, that is painful to look at. If possible I would like to avoid Django-specific code in pdoc though. It won't take long before we are also looking at Flask, SQLAlchemy, FastAPI, etc - that exceeds the scope of pdoc. I think this is another good example in favor of accepting doc.Module objects for pdoc.pdoc, because then you can do all those changes before passing the module objects to pdoc. Most properties in pdoc are intentionally implemented as cached_property so that you can just modify/override them before render (although you need to be careful about order if cached accessors depend on other cached accessors). I'm thinking of something like this:

module_names = extract.walk_specs(modules)
module_names = exclude_django_boilerplate_mods(all_modules)

modules = [
    doc.Module(extract.load_module(mod))
    for mod in module_names
]
remove_django_specific_boilerplate(modules)

pdoc.pdoc(*modules, output_directory=...)

Model field docstrings don't work as expected.

Yes, see above. Having some custom code that properly fixes up doc.Module objects sounds like the way to go to me. I'd be happy to merge a PR that makes pdoc.pdoc accept pdoc.doc.Module objects as part of the modules argument (it may make sense to implement that as part of walk_specs, I haven't looked closely). :)

[odigity]

Let me preface this by saying that pdoc targets library documentation and not really Django apps if that makes sense.

Do you know of a tool better suited to the purpose of documenting a Django project?

Please don't say Sphinx. :)

[mhils]

I've never worked much with Django, so I can't say much on that. Honestly you may be best off just writing a bit of high-level/architecture prose and let everyone else browse the sources. That or Sphinx, which is more complex, but also more powerful than pdoc. :)

[odigity]

Thanks for the feedback. Given the current options, we've decided to just write docs by hand in a wiki rather than using any tooling to extract inline content. I'll close my issues now.

[mhils]

I think that's a perfectly fine approach, thanks again for the detailed feedback! 🍰