deephaven / deephaven-plugins

Deephaven Plugins
11 stars 15 forks source link

EPIC: Improve plotly-express plugin to use table operations directly whenever possible #111

Open jnumainville opened 1 year ago

jnumainville commented 1 year ago

This comes out of some discussions with Colin as how to best handle figure updates.

Currently, lots of logic is calculated in python outside of tables, rather than in terms of python applied to table transformations. For example, for a partitioned table, state is pulled from the table and frozen to calculate plot_bys, but table updates can rarely cause data races. This was not really a problem with simple tables, but now can be a problem with partitions, and if a constituent table is empty, it will be ignored, but this is an ugly solution.

Converting the logic to table transformations has the side effect of being a useful way to compute chart deltas, as messages can be created to just send the new figure data. Currently, the whole figure is sent, which is not optimal, as often the only difference is a new trace in the data object.

Breakdown to come when this can be prioritized, but will involve refactoring chunks of code to operate on table operations and defining how deltas should be sent.

mattrunyon commented 1 year ago

I'd say sending the whole figure is probably not an issue given the size of the figure is relatively small and the frequency that we'd expect it to update is relatively low. The client already handles subscribing to only new tables, so the small bit of JSON is the only thing sent that is duplicated

Implementing deltas for the figure definition might be more effort and complexity than it is worth

jnumainville commented 1 year ago

I do generally agree it's possibly not worth doing given the likely update frequency, and is a question of how much resources we want to dedicate towards optimization.