search

posit-dev / py-shinywidgets

Render ipywidgets inside a PyShiny app
MIT License
46 stars 5 forks source link

Support binary data over `ShinyComm` #151

Open manzt opened 3 months ago

manzt commented 3 months ago

From https://github.com/manzt/quak/issues/30

Many anywidgets and several traditional Jupyter Widgets rely on Jupyter's support for binary data over the comm.

The message protocol allows sending messages including both JSON serializable and binary blobs together.

Right now buffers is ignored in ShinyComm.send:

https://github.com/posit-dev/py-shinywidgets/blob/d86f74b0f3eb9f03b9698249346bae8558b24f37/js/src/comm.ts#L35

~and on_msg callback should take a second buffers?: Array<ArrayBuffer> argument:~

Of course, shiny doesn't need to implement the Jupyter Widgets Protocol... but I wasn't able to figure out how to pack both JSON-serializable & binary blobs together. Maybe msgpack might be of interest, as long as Shiny as way to send a binary input blob.

EDIT: Taking a closer look, shinywidgets does support sending binary data to the front end (although it does so with base64 encoding the data). I've made a PR #152 to ensure the types are what are expected by widgets.

manzt commented 3 months ago

It might be useful to throw an error if front-end code tries to ShinyComm.send buffers, since would help end users quickly identify that this is a limitation of shinywidgets support for Jupyter Widgets. That would help avoid end users reporting issues to the widget libraries when buffers are silently deleted.