Customizing Widget content and adding custom widgets to `._components`

[grepler]

• pydantic-panel version: 0.1.13
• Python version: 3.10.9
• Operating System: Mac OS 12.4

Description

I am utilizing this awesome project to interact with SQLAlchemy models and save changes from Panel webapps back to my sqlite database.

I'm using SQLModel, which integrates SQLAlchemy and Pedantic into a common base class, then pydantic-panel to get easy-to-use Panel interactivity.

I was able to achieve my goal, but it seems very hacky and I wonder if there's a more elegant way to go about this. My primary purpose is to separate out the business logic / models from the Panel UI elements, and this project seems to allow it nicely. (It's very nice to be able to throw up an interactive and persistent settings box in Jupiter just by typing pn.panel(sqlmodel_obj), and the dream is to then also easily transform these data-exploration notebooks into internal user-facing tools once the process development/exploration is done).

Out of the box functionality

The recursive nature of pedantic-panel means that things work out of the box, but my goal is to have control over what and how things are shown.

See in the below screenshot how, when using the default automatic class identification, all fields on the SQLModel/pydantic model are shown and editable:

Customizing Editors

Using a new class DataFieldEditorCard(PydanticModelEditorCard) I was able to customize the output, like so:

Question

What is the best way to instantiate a Panel Widget / Layout with customized Columns / Rows, etc., either overriding the automatic rendering of specific fields or adding new, wholly custom component widgets?

What I Did

I wrote an __init__ method on my custom editor, basing it off of PydanticModelEditorCard. I think it's pretty gross and there has to be a better way that I'm just not seeing.

Hiding Fields

From what I can tell from reading the codebase, the key class is PydanticModelEditor, which has some important attributes:

• .fields can be passed into the super's __init__, and is used by .widgets() to filter the list of pedantic fields which are added to the widgets. This seems to work fine for me. I'm pretty okay with this.

Adding new components

On the other hand, since the ._recreate_widgets() function is the only place where the self._widgets list is defined / updated, and then in the subsequent line the self._components array has it's contents replaced, I am struggling to efficiently add new widget components to the generated Panel object.

https://github.com/jmosbacher/pydantic-panel/blob/4cbcd380037df28476a4c7c4911994a6f246ff6f/pydantic_panel/widgets.py#L214-L217

I have had to:

1. create an internal attribute on the Editor (._selected_roles) to store my custom widget,
2. insert the widget into the created widget array,
3. then finally replace the contents of self._components.

Here is the code I am currently using:

from pydantic_panel import infer_widget
from pydantic_panel.widgets import PydanticModelEditorCard
from pydantic.fields import FieldInfo, ModelField
from typing import Optional, ClassVar, Type, List, Dict, Tuple, Any

import pandas as pd

import param
import panel as pn
from panel.layout import Column, Divider, ListPanel, Card
from panel.layout.gridstack import GridStack

from .models import DataField, Dataset, DataRole
### CONTEXT: there is some logic from these objects which the custom MultiChoice widget uses to fetch allowed options for the field.

class DataFieldEditorCard(PydanticModelEditorCard):
    """Same as PydanticModelEditor but uses a Card container
    to hold the widgets and synces the header with the widget `name`
    """

    _composite_type: ClassVar[Type[ListPanel]] = pn.Card
    collapsed = param.Boolean(False)
    collapsible = param.Boolean(False)

    is_json = param.Boolean(False)
    value: DataField = param.ClassSelector(class_=DataField)

   ### STORING CUSTOM WIDGET ###
    _selected_roles: pn.widgets.MultiChoice = param.ClassSelector(class_=pn.widgets.MultiChoice)

    def __init__(self, **params):
        super().__init__(**params)
        print(f'instantiating DataFieldEditorCard: {params=}')
        print(self.value)

        self._selected_roles = pn.widgets.MultiChoice(
            name='ROLES', 
            value=self.value.roles,
            options={x.name: x for x in self.value.dataset.roles_available}
            )

        self._composite.header = self.name
        self.link(self._composite, name="header")
        self.link(self._composite, collapsed="collapsed")

        ### THIS IS WHERE I SHOEHORN MY CUSTOM WIDGET INTO THE EDITOR ###
        self._widgets['roles'] = self._selected_roles
        self._composite[:] = self.widgets

    @param.depends("_selected_roles.value", watch=True)
    def _update_value(self, value=None):
        """
        Whenever the Custom Widget is updated, update the SQLModel and check if the available role options have changed.
        This will also be cleaned up, at the moment relationships are being destroyed and recreated, causing some confusion for SQLAlchemy
        """
        if self.value is None or self._selected_roles is None:
            return

        self.value: DataField
        self.value.roles.clear()
        self.value.roles.extend(self._selected_roles.value)
        # update the set of available options
        self._selected_roles.options = dict(self.value.dataset.roles_available)

@infer_widget.dispatch(precedence=2)
def infer_widget(value: DataField, field: Optional[FieldInfo] = None, **kwargs):
    """
    Dispatcher for DataField Pydantic types.
    """
    # discovered that this seems to be necessary in: 
    # https://github.com/jmosbacher/pydantic-panel/blob/4cbcd380037df28476a4c7c4911994a6f246ff6f/pydantic_panel/widgets.py#L648
    class_ = kwargs.pop("class_", type(value))

    name = kwargs.pop("name", value.name)

    return DataFieldEditorCard(
        name=value.name, 
        value=value, 
        class_=class_, 
        fields=list(('name', 'roles', 'description')),
        **kwargs
        )

[jmosbacher]

@grepler great to hear that you have found this package useful.

I am not sure I fully understood what you are trying to achieve but perhaps you are looking for the typing.Literal type? This type is supported by str and list dispatchers, so if e.g. your roles attribute of DataField is of type List[Literal["DOCDATE","OTHER_ROLE","YET_ANOTHER_ROLE"]] then the MultiChoice widget will be used to edit it by default. If on the other hand you have a dynamic list of available roles then I dont really see any way of supporting that in the framework as the way to fetch the list of roles would be different for every user. So in that case I think your solution is fine. That being said, if you post your pydantic models then maybe I will see a better way to do it but not sure currently where the data is coming from.

[grepler]

Ultimately my goal is to keep the sqlalchemy models and panel UI components separate, and the dispatch method works very nicely. It may be an unintended use of your package, but the core dispatch and synchronization functionality makes it very appealing to leverage your existing work.

I have published my proof-of-concept repo here, with a Jupiter notebook walking through the models, and my custom editor: https://github.com/grepler/sqlmodel-panel-poc

@MarcSkovMadsen sorry to ping you, I have followed your projects around Panel as I have tried to get up-to-speed. I think that my request is similar to your comment on https://github.com/jmosbacher/pydantic-panel/issues/4#issuecomment-1200423406. That is, using editors to supply not just a single editable param entity, but an entire, customized widget.

Have you had any success / best practices for separating business logic and UI, while still preserving the pn.panel(model_obj) ability which is so nice here?

[jmosbacher]

Thanks for the link, I think I understand a bit better what you are trying to achieve. I would be happy to accept a PR on this but it would have to support nested widget definition since fields can be nested, perhaps something like the traitlets.config nested structure. BTW I noticed that you want to validate entire columnar structures and not just single rows independently, so you may want to take a look at the pandera package. It may be a better fit for your use case when defining the actual validation functions and its completely compatible with pydantic models.

[grepler]

Closing this issue, thanks for those links @jmosbacher!

After I spent some more time with the package I figured out how to do it properly, using the dispatcher.

Would you accept a pull-request on this package to mark specific attributes as read-only?

[jmosbacher]

@grepler thats great to hear. I would be happy to accept any PR that improves usability. I am currently focused on writing my thesis so have very little time for development myself but am happy to make some time to review a PR.