dynamic module introspection doesn't render classes

I'm trying to dynamically document modal.functions.web_endpoint. Here is the yaml

project:
  type: website

website:
  title: "."
  navbar:
    left:
      - href: index.qmd
        text: Home
      - about.qmd

format:
  html:
    theme: cosmo
    css: styles.css
    toc: true

# tell quarto to read the generated sidebar
metadata-files:
  - _sidebar.yml

quartodoc:
  # the name used to import the package
  package: modal

  # write sidebar data to this file
  sidebar: _sidebar.yml

  sections:
    - title: Documentation utilities
      desc: Utilities for documentation.
      dynamic: true
      contents:
        - functions.web_endpoint

In Ipython you can see the help just fine

In [4]: from modal.functions import web_endpoint

In [5]: ?web_endpoint
Signature:
web_endpoint(
    method: str = 'GET',
    label: Optional[str] = None,
    wait_for_response: bool = True,
) -> Callable[[Callable[..., Any]], modal.functions._PartialFunction]
Docstring:
Register a basic web endpoint with this application.

This is the simple way to create a web endpoint on Modal. The function
behaves as a [FastAPI](https://fastapi.tiangolo.com/) handler and should
return a response object to the caller.

Endpoints created with `@stub.web_endpoint` are meant to be simple, single
request handlers and automatically have
[CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) enabled.
For more flexibility, use `@stub.asgi_app`.

To learn how to use Modal with popular web frameworks, see the
[guide on web endpoints](https://modal.com/docs/guide/webhooks).

All webhook requests have a 150s maximum request time for the HTTP request itself. However, the underlying functions can
run for longer and return results to the caller on completion.

The two `wait_for_response` modes for webhooks are as follows:
* `wait_for_response=True` - tries to fulfill the request on the original URL, but returns a 302 redirect after ~150s to a result URL (original URL with an added `__modal_function_id=...` query parameter)
* `wait_for_response=False` - immediately returns a 202 ACCEPTED response with a JSON payload: `{"result_url": "..."}` containing the result "redirect" URL from above (which in turn redirects to itself every ~150s)
File:      /opt/anaconda3/lib/python3.9/site-packages/modal/functions.py
Type:      function

However nothing is rendered in quartodoc

To repro, checkout the modal branch of this repo, and run quartodoc build followed by quarto preview

quartodoc doesn't support dynamic being set to true on the section (though it could!), and pydantic unfortunately is happy to accept extra fields and then discard them 😬.

Does it work if you set dynamic at the top-level (right under quartodoc) or for the individual content entries?

https://machow.github.io/quartodoc/get-started/basic-docs.html#top-level-options

https://machow.github.io/quartodoc/get-started/basic-content.html#dynamic-lookup

I wonder if we can set pydantic to be strict and not silently drop extra fields 🤔

If I set dynamic: true on the invidiual content entry, it still doesn't render properly. Concretely, it doesn't show the signature

project:
  type: website

website:
  title: "."
  navbar:
    left:
      - href: index.qmd
        text: Home
      - about.qmd

format:
  html:
    theme: cosmo
    css: styles.css
    toc: true

# tell quarto to read the generated sidebar
metadata-files:
  - _sidebar.yml

quartodoc:
  # the name used to import the package
  package: modal
  # write sidebar data to this file
  sidebar: _sidebar.yml

  sections:
    - title: Documentation utilities
      desc: Utilities for documentation.
      contents:
        - functions
        - name: web_endpoint
          contents: [functions.web_endpoint]
          dynamic: true

Ah, web_endpoint is created too dynamically for the relatively simple way we dynamically load docstrings. If they used decorators, then it would work...


# Currently unsupported, doesn't know web_endpoint is a function ----

@typechecked
def _web_endpoint(*args, **kwargs): ...

web_endpoint = synchronize_api(_web_endpoint)

# Supported ----

@synchronize_api
@typechecked
def web_endpoint(*args, **kwargs): ...

It looks like a special tool for dynamic loading, called the griffe Inspector can load it. Ideally, we shift to using this for dynamic loading (and contribute upstream for any funky cases):

from griffe.agents.inspector import inspect

mod = inspect("modal.functions")

# griffe Function object
obj = mod.members["web_endpoint"]

obj.docstring.value

Note that AFAICT the griffe Inspector doesn't work for packages like plotnine, so we may need to do a bit of research:

# this took ~15 seconds to run on my laptop
mod = inspect("plotnine")

"ggplot" in mod.members      # False

# It seems to be in the correct submodule though
# it looks like the griffe inspector may not do Alias objects
mod = inspect("plotnine.ggplot")
mod.members["ggplot"]

There are lots of objects in plotnine so yeah, loading every module in memory and inspecting every member is quite demanding. A very small part of the logs (I added a print statement in the inspector):

Click to expand

``` inspecting plotnine.themes.themeable.strip_text_y inspecting plotnine.themes.themeable.strip_text_y.__annotations__ inspecting plotnine.themes.themeable.strip_text_y.__class__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__abstractmethods__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__annotations__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__bases__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__basicsize__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__call__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__delattr__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__dict__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__dictoffset__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__dir__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__doc__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__eq__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__flags__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__format__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__ge__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__getattribute__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__getstate__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__gt__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__hash__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__init__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__init_subclass__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__instancecheck__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__itemsize__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__le__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__lt__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__module__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__mro__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__name__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__ne__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__new__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__or__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__prepare__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__qualname__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__reduce__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__reduce_ex__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__repr__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__ror__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__setattr__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__sizeof__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__str__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__subclasscheck__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__subclasses__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__subclasshook__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__text_signature__ inspecting plotnine.themes.themeable.strip_text_y.__class__.__weakrefoffset__ inspecting plotnine.themes.themeable.strip_text_y.__class__.mro inspecting plotnine.themes.themeable.strip_text_y.__delattr__ inspecting plotnine.themes.themeable.strip_text_y.__dict__ inspecting plotnine.themes.themeable.strip_text_y.__dir__ inspecting plotnine.themes.themeable.strip_text_y.__doc__ inspecting plotnine.themes.themeable.strip_text_y.__eq__ inspecting plotnine.themes.themeable.strip_text_y.__format__ inspecting plotnine.themes.themeable.strip_text_y.__ge__ inspecting plotnine.themes.themeable.strip_text_y.__getattribute__ inspecting plotnine.themes.themeable.strip_text_y.__getstate__ inspecting plotnine.themes.themeable.strip_text_y.__gt__ inspecting plotnine.themes.themeable.strip_text_y.__hash__ inspecting plotnine.themes.themeable.strip_text_y.__init__ inspecting plotnine.themes.themeable.strip_text_y.__init_subclass__ inspecting plotnine.themes.themeable.strip_text_y.__le__ inspecting plotnine.themes.themeable.strip_text_y.__lt__ inspecting plotnine.themes.themeable.strip_text_y.__module__ inspecting plotnine.themes.themeable.strip_text_y.__ne__ inspecting plotnine.themes.themeable.strip_text_y.__new__ inspecting plotnine.themes.themeable.strip_text_y.__reduce__ inspecting plotnine.themes.themeable.strip_text_y.__reduce_ex__ inspecting plotnine.themes.themeable.strip_text_y.__repr__ inspecting plotnine.themes.themeable.strip_text_y.__setattr__ inspecting plotnine.themes.themeable.strip_text_y.__sizeof__ inspecting plotnine.themes.themeable.strip_text_y.__str__ inspecting plotnine.themes.themeable.strip_text_y.__subclasshook__ inspecting plotnine.themes.themeable.strip_text_y.__weakref__ inspecting plotnine.themes.themeable.strip_text_y._hierarchy inspecting plotnine.themes.themeable.strip_text_y._registry inspecting plotnine.themes.themeable.strip_text_y.apply inspecting plotnine.themes.themeable.strip_text_y.apply_ax inspecting plotnine.themes.themeable.strip_text_y.apply_figure inspecting plotnine.themes.themeable.strip_text_y.blank_ax inspecting plotnine.themes.themeable.strip_text_y.blank_figure inspecting plotnine.themes.themeable.strip_text_y.from_class_name inspecting plotnine.themes.themeable.strip_text_y.is_blank inspecting plotnine.themes.themeable.strip_text_y.merge inspecting plotnine.themes.themeable.strip_text_y.order inspecting plotnine.themes.themeable.strip_text_y.rcParams inspecting plotnine.themes.themeable.strip_text_y.registry inspecting plotnine.themes.themeable.strip_text_y.setup_figure inspecting plotnine.themes.themeable.subplots_adjust inspecting plotnine.themes.themeable.subplots_adjust.__annotations__ inspecting plotnine.themes.themeable.subplots_adjust.__class__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__abstractmethods__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__annotations__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__bases__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__basicsize__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__call__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__delattr__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__dict__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__dictoffset__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__dir__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__doc__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__eq__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__flags__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__format__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__ge__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__getattribute__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__getstate__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__gt__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__hash__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__init__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__init_subclass__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__instancecheck__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__itemsize__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__le__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__lt__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__module__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__mro__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__name__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__ne__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__new__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__or__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__prepare__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__qualname__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__reduce__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__reduce_ex__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__repr__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__ror__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__setattr__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__sizeof__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__str__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__subclasscheck__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__subclasses__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__subclasshook__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__text_signature__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.__weakrefoffset__ inspecting plotnine.themes.themeable.subplots_adjust.__class__.mro inspecting plotnine.themes.themeable.subplots_adjust.__delattr__ ```

% python -c 'from griffe.agents.inspector import inspect; inspect("plotnine")' | wc -l
253418

Maybe we could drastically reduce this if we didn't inspect __class__ and its potential __bases__. But IMO that's just the disadvantage of the inspector over the visitor.

Hmmm, we might find some possible optimizations though:

inspecting plotnine.geom_area
inspecting plotnine.geom_area.DEFAULT_AES
inspecting plotnine.geom_area.DEFAULT_PARAMS
inspecting plotnine.geom_area.NON_MISSING_AES
inspecting plotnine.geom_area.REQUIRED_AES
inspecting plotnine.geom_area.__annotations__
inspecting plotnine.geom_area.__base__
inspecting plotnine.geom_area.__base__.DEFAULT_AES
inspecting plotnine.geom_area.__base__.DEFAULT_PARAMS
inspecting plotnine.geom_area.__base__.NON_MISSING_AES
inspecting plotnine.geom_area.__base__.REQUIRED_AES
inspecting plotnine.geom_area.__base__.__annotations__
inspecting plotnine.geom_area.__base__.__base__
inspecting plotnine.geom_area.__base__.__base__.DEFAULT_AES
inspecting plotnine.geom_area.__base__.__base__.DEFAULT_PARAMS
inspecting plotnine.geom_area.__base__.__base__.NON_MISSING_AES
inspecting plotnine.geom_area.__base__.__base__.REQUIRED_AES
inspecting plotnine.geom_area.__base__.__base__.__annotations__
inspecting plotnine.geom_area.__base__.__base__.__class__
inspecting plotnine.geom_area.__base__.__base__.__class__.__abstractmethods__
inspecting plotnine.geom_area.__base__.__base__.__class__.__annotations__
inspecting plotnine.geom_area.__base__.__base__.__class__.__bases__
inspecting plotnine.geom_area.__base__.__base__.__class__.__basicsize__
inspecting plotnine.geom_area.__base__.__base__.__class__.__call__
inspecting plotnine.geom_area.__base__.__base__.__class__.__class__
inspecting plotnine.geom_area.__base__.__base__.__class__.__class__.__abstractmethods__
inspecting plotnine.geom_area.__base__.__base__.__class__.__class__.__annotations__
inspecting plotnine.geom_area.__base__.__base__.__class__.__class__.__bases__
inspecting plotnine.geom_area.__base__.__base__.__class__.__class__.__basicsize__

Actually the just-merged inheritance support feature will help :slightly_smiling_face:

With Griffe installed from main branch:

% python -c 'from griffe.agents.inspector import inspect; inspect("plotnine")' | wc -l
5995

Ah, thanks! I've been digging more into how griffe and sphinx do dynamic inspection, and should have more bandwidth to contribute to the dynamic inspect stuff after scipy next week!

machow / quartodoc

dynamic module introspection doesn't render classes #198