deneb-viz / deneb

Deneb is a custom visual for Microsoft Power BI, which allows developers to use the declarative JSON syntax of the Vega or Vega-Lite languages to create their own data visualizations.
https://deneb-viz.github.io
MIT License
184 stars 15 forks source link

Treat Data Changes as Changesets in the Vega View Rather than Refreshing It #410

Open PBI-David opened 9 months ago

PBI-David commented 9 months ago

Hi Daniel - I have created the attached force directed graph but interactions with Power BI are currently limited.

Thumb

Problem 1: If you enable cross filtering (selection) of data points (option below) then this works to an extent but if you actually select a node to cross filter another visual, then the whole simulation is retriggered, presumably because of an upstream dataset change. This means that cross filtering is not really workable with force directed graphs currently. To recreate, enable the option and then click a node.

image

Problem 2: You can see some sliders at the bottom which change signals. These are Vega sliders and I would like to replace them with Power BI slicers. This is related to problem 1 in that a change to a parameter affects the whole dataset and triggers a simulation restart. i.e. if you zoomed in and then used an external slicer to change the node size, the whole view would reset and the simulation would be restarted which looks terrible.

The first problem I believe is internal and related to how you currently implement the selection logic but no idea to be honest. The second problem I think is actually a feature request. What I envisage for the second is perhaps a second field well where you could place slicer measures which don't trigger a whole dataset refresh of the first field well. Perhaps called dataset2 or something which is isolated from dataset1.

Hopefully, these make sense but if not, let me know and I will try to clarify.

Force Directed Graph.pbix.zip

dm-p commented 9 months ago

I think that both points are fundamentally the same thing underneath.

Due to issues with how the Vega view embedding works with React, we treat the dataset as volatile, so it's fully patched into the spec behind the scenes before parsing and rendering. Here's the complete list of things that we regard as such.

Up until 1.6, Deneb would supply the dataset as a separate object, which is technically how it should have been done, and this caused us the sporadic and random errors we sometimes saw before 1.6. Ultimately, it came down to the embed not making the data available to the spec at the right time, and some aspects of the parsing process needed something to work with.

In your example (which is awesome, BTW), we can see the effect of changing something that we regard as non-volatile, such as one of the properties in the pane. Anything in the Advanced editor menu is a good example. Property changes trigger an update to the visual, but as they aren't regarded by our internal logic as volatile for the presentation of the spec, changes will stick, e.g.:

https://github.com/deneb-viz/deneb/assets/10572054/576bebbe-64f4-4297-a8b3-a1985f80b8fb

The last property we change requires us to update Vega's schemes for the Discrete ordinal colors binding, so this will re-trigger the render, but you can see that the others preserve the current view.

Where I'm going with this is that we can explore the following for the dataset as an idea:

The thing with changesets is that they require us to re-run the view anyway, so we'll need to see if this resets the current output. If not, then it could be an option.

If this is a viable approach, we'll also need to look at the window-based fetching for datasets > 10,000 rows, as this replaces the visual with a status message, which would require us to re-render the whole thing. We could change this to a floating message over the current view rather than replacing the current DOM.

dm-p commented 9 months ago

I did a quick test where I used a button to extract the current dataset from the view, slice it, and pass it back in.

https://github.com/deneb-viz/deneb/assets/10572054/dcacf315-094e-4cfd-8143-ed24285c1276

Updating the view programmatically after the data change appears to keep the value of a bound signal as it was prior to the change. It will also preserve the scroll position if the container overflows, which is encouraging.

This should mean that the proposed approach in the previous comment is viable. It will be a decent-sized change to the current workflow, but I think that this can be something we look at for 1.7.

PBI-David commented 9 months ago

Thanks Daniel. I don't completely understand everything you've written but will gladly follow your lead and test when the time is right. To be 100% clear, I have made a video below of the problem 1 issue.

https://github.com/deneb-viz/deneb/assets/86846675/f235a532-860f-4f10-9b51-5720f40efa6a

Giammaria commented 9 months ago

Glad I'm not the only one that didn't fully understand that 😀

This is just more evidence for how complex everything is behind the scenes with Deneb and the Vega Viewer. I'll plan to re-read after my eyes uncross a bit, as I'd like to (slowly) become more familiar with the challenges surrounding implementing changes like this.