search

deephaven / deephaven-plugins

Deephaven Plugins
11 stars 15 forks source link

Handle document delta updates sent from the server #123

Open mofojed opened 1 year ago

mofojed commented 1 year ago

Currently for every update, the server sends the entire re-rendered document, even if only a part of it has changed. Should only send the diff between the current document and the new document, to decrease payload size.

Note this is separate from #129 , where we only send newly exported objects.

mofojed commented 1 week ago

Take a look at how JSON diffs are created and applied. Might be beneficial to look at how JSON pointers work: https://datatracker.ietf.org/doc/html/rfc6901

mofojed commented 1 week ago

@wusteven815 JSON patch is the spec we should implement for processing the deltas: https://jsonpatch.com/ Looks like that page links a few JS libraries that implement it, so would want to see which one is most appropriate. Seems to only be one Python JSON patch, so hopefully it does what we need to (or we can extend).

wusteven815 commented 5 days ago

The fast-json-path library (https://www.npmjs.com/package/fast-json-patch) has all the features we need and has a lot of weekly downloads.

mofojed commented 5 days ago

On the server side, we'll want to keep track of the previous encoded_document and get a delta with the current one. If previous doesn't exist, send documentUpdated, if it does exist, send documentDelta instead with the JSON patch: https://github.com/deephaven/deephaven-plugins/blob/b3f545942c68921f2dbf6c4142c6e7b635b23907/plugins/ui/src/deephaven/ui/object_types/ElementMessageStream.py#L450 On the client side, we'll need to add a new jsonClient method to handle the documentDelta: https://github.com/deephaven/deephaven-plugins/blob/b3f545942c68921f2dbf6c4142c6e7b635b23907/plugins/ui/src/js/src/widget/WidgetHandler.tsx#L234 And apply the patch to the previous document.