Replace traitlets with pydantic?

[davidbrochart]

Currently, ipywidgets is based on traitlets for validating and observing data. Pydantic is a modern library for data validation. Could Pydantic replace traitlets in ipywidgets?

[davidbrochart]

For the observer pattern, could something like https://github.com/davidbrochart/pyceptive work?

[vidartf]

Given how ingrained traitlets are into the ipywidgets datastructures, I don't think it is very realistic to replace it. If it was to even be considered, this issue would need to also include some sort of a motivation that is large enough to warrant the time, effort and pain.

[mangecoeur]

Just dropping by to say this is an attractive idea for making data models that play nice with dataclasses (supported by Pydantic and SQLAlchemy) as well as widgets. My example is making a data model class that can be bound to a UI widget but also be returned from a FastAPI endpoint or stored in a database.

A possible strategy would be to somehow build dataclass support into traitlets itself in a way that traitlet derived classes would play nice with other libraries that also support dataclasses, including Pydantic. This would allow at least a transitional mode even if you end up with slightly barbaric things like foo: float = Float(). AFAIU a lot of the compatibility story is mostly to do with what metadata is associated with a class, as long as people respect duck-typing, if the right metadata is generated then it doesn't matter if it came from a dataclass decorator or if it's generated by traitlets or anything else.

[tlambert03]

just wanted to say this is an idea i think about a lot as well. I have a small library (psygnal) that focuses only on implementing the observer pattern. There is a pydantic subclass that adds events and more recently I've been trying to abstract that dependency to support a variety of dataclass-like implementations (including stdlib, pydantic, attrs, msgspec) (docs here). It's also what backs magicgui (which itself uses ipywidgets in a jupyter environment).

anyway, +1 from me. I'd also love to see a more modern dataclass/observer pattern backing ipywidgets.

[davidbrochart]

Thanks for sharing @tlambert03, very nice. I had also played with this idea in pyceptive, but psygnal looks much better. I think it would be interesting to experiment with these new ideas in ypywidgets, since we don't use the observer pattern there yet.

[tlambert03]

cool, thanks for pointing me to that! (y-py, and indeed yjs, are new to me)

I'll also mention anywidget by @manzt here – where we've been battling recently with some similar observer/comm design pattern decisions in the general realm of jupyter widgets.

nice to make your acquaintance :)

[timkpaine]

@davidbrochart I know you're working on something similar with yjs, but ive started taking a look at building a framework on top of pydantic to work both inside and outside of widgets for my ipydagred3 library. Completely separate from Jupyter, though useable easily inside ipywidgets. My main motivation is a desire to use ipydagred3 outside of Jupyter in a vanilla web app (which means abstracting away comms completely and being able to swap in different transport protocols like e.g. vanilla websockets, maybe inside of a web framework).

It is a bit more manual than pyceptive for now, but similar idea. As an example, with ipydagred3 I do a lot of manual synchronization, so this framework would work a bit like that (though I have thought about recursively replacing setattr, setitem, etc on all the elements to enable fully transparent synchronization, maybe down the road). I'm also hoping that pydantic-core moves quickly, as ideally one could use the same rust-based model framework in both python and JS via wasm.

Once it's a bit more stable and a bit less embarrassingly bad, im happy to share/collaborate if this is of interest to anyone else.

[maartenbreddels]

My main motivation is a desire to use ipydagred3 outside of Jupyter in a vanilla web app (which means abstracting away comms completely and being able to swap in different transport protocols like e.g. vanilla websockets, maybe inside of a web framework).

Hi Tim, we are flying a bit under the radar right now, but Solara allows running ipywidgets (see https://solara.dev/docs/tutorial/ipywidgets ) in a regular web framework like starlette/fastapi or flask.

[davidbrochart]

Great to see all these efforts around widgets! I see we are all making progress in different directions, maybe we could present our work in one of Jupyter Widgets' meetings someday? I'm particularly interested in using Ypy/Yjs with widgets these days, as I think CRDTs will become central in JupyterLab's architecture in the future. Using these three libraries:

• reacttrs: reactive attributes extracted out from Textual.
• ypywidgets: Python side that uses Ypy over kernel Comm.
• yjs-widgets: TypeScript side that uses Yjs over Comm.

I've been able to reimplement ipywidgets' 2-way communication. I think it's a fairly simple implementation, and it makes widgets truly collaborative (ipywidgets' architecture is not conflict-free).

https://user-images.githubusercontent.com/4711805/229904257-ecf1bbfa-09cc-4023-8121-1cb44860f2f4.mp4