Bind Vega signals to the Visualization Pane inputs (a dream)

[avatorl]

I don't know if it will ever be possible in any custom visual (or if Deneb will ever become a native visual, or if it's even possible in a native visual). But it will significantly simplify the way how custom visualizations for Power BI are being created and it will deeply integrate Vega into Power BI.

The idea is in binding Vega signals to the corresponding Visualization pane controls.

Basically, if there is Vega signal

    {
      "name": "colorTitle",
      "value": "#333333"
    }

Then a color selector should automatically appear on the Format Visual tab of the Visualization pane. An fx button for Measure driven conditional formatting should appear there as well.

I assume Vega signal will need "element" property to be defined.

And Deneb visual should create corresponding input elements on the Visualization pane for all Vega signals with "element" property defined.

As a result any Vega visualization inside of Deneb will act like an unique custom visual with its own Visualization pane settings. Any Power BI Desktop user will be able to control formatting settings of the Vega visualization using native Power BI Visualization pane visual interface. There will be no need in editing Vega code just to change formatting settings (assuming there were signal created for all important properties of the Vega visualization). It will be easy to fully control Vega visualization from other visuals/slicers on the page (via DAX measures bound to the Vega signals).

[Giammaria]

This would be a nice feature. It may be tricky to implement though since the objects property in the capabilities.json file for Deneb needs to be defined ahead of time. I guess one could predefine those properties with the common properties defined in a Vega spec’s config section. Then I imagine there would need to be some sort of naming convention either in the param/signal name or in its description to be able to tie the signal value to the appropriate power bi formatting option. You could have only those properties that have a matching signal defined in the spec show up in the formatting pane. In this scenario the spec authors would need to know these naming conventions in order to use this functionality. Pretty significant L.O.E if I had to guess, but would be really useful for sharing templates with report developers that don’t have the bandwidth to learn the Vega languages.

[dm-p]

I already provided a thought-based reply for this on the Twitter thread where it originated, but it's been playing on my mind as a concept, particularly as I was thinking of doing custom signals in v2. I've written about these somewhere as an idea but can't find where, but essentially this was centred around the concept of reserving named properties in the formatting pane that could have values or measures bound to them that would create named signals in the Vega view with the property values. The challenge around these is that they can only be textual or numeric, as conditional formatting doesn't support other types, and we would have to explicitly reserve n properties (so if you want n + 1, this is not really possible without adding the capability for it, rebuilding the visual and re-publishing it).

@Giammaria has already noted the same thing as the tweet in that you conventionally have to declare your objects and properties in the visual's capabilities so that Power BI has a list of valid properties for your visual, e.g.:

You can absolutely declare a dynamic property at run time, but all it does is create a control in the pane. This can be whatever you specify via code (as you need to write code to choose the correct controls etc. for the specified names, and some logic to populate the pane). We could call this something like dynamicProperty, and this would render for the user, e.g.:

So, realistically... this bit we could do. We might even be able to look at the element type from the binding (V | VL) if we had enough time, and create sliders or dropdowns or whatever.

Here's where the idea breaks down though (as we might expect). Because dynamicProperty is not a declared valid property in the visual capabilities, if we change the value, the main window does not store it in the visual metadata, and the value is therefore never passed to the visual when the pane is re-evaluated on the next update (when values, if different from their defaults are made available to the visual logic) :(

We can't even use the persist property API to forcibly push it into the visual's objects in the main window, as Power BI discards unknown property names (I've tested with a known property also and that gets updated remotely, as expected).

I'm happy to make a feature request to MS to see if it's possible to allow dynamic property declaration for this use case, but I'm pretty sure there will be a reason why this isn't already possible. But I can appreciate and understand the use case, so if I hear anything back, I'll be sure to try and update this.

[dm-p]

I've raised an idea for this with the custom visuals folks, so (as stated in the previous comment), I'll let you know if we get any traction with it.

[avatorl]

Thank you Daniel,

It will be really great if they can allow dynamic property declaration. I have had a meeting recently with a potential client who wanted a bunch of highly customized data visualizations and Vega is perfect tool to make them. But what stops him from using Deneb-Vega is lack of Visualization Pane controls to edit visualization settings. So, I'm sure this feature is a real game changer.

but I'm pretty sure there will be a reason why this isn't already possible In this case, as I understand, possible workaround is to create clones of Deneb visual with pre-defined capabilities (for each Vega visualization). I understand that I need to edit capabilities.json to add properties to Visualization pane. But how hard is it to bind Vega signals to these pre-defined properties? Do you have any example of how to do such binding?

[dm-p]

It's not hard per se, but it is a lot of work. If you were looking to do this, the high-level approach might be:

• You have to add the capabilities
• You need to explicitly declare these properties for the settings classes
  • If you are creating new property menus then you need to declare new classes for these and their properties.
  • So, there are two sides to properties: declaring their placeholders in capabilites and declaring code to set and process their defaults from the visual side so that they are evaluated and created in the pane.
  • It's a bit more complex than this, but I'm going for brevity. The doc has more details on this.
  • Note that Deneb v1 uses the conventional enumerateObjectInstances method, which has been the standard way until the new formatting cards were recently introduced. v2 will use the getFormattingModel method (which is how formatting cards work).
  • The underlying principle applies to both methods though - capabilities must be declared and there must be corresponding logic to populate these from within the code. It's why properties are such an administrative burden to add and maintain.
• With this in-place, you will need to update the code to inject these properties into the Vega view as named signals.
  • You would do this using the Vega View API.
  • Deneb does this by leveraging the onNewView method of react-vega.

Note that if you clone the visual you will also need to update its guid in pbiviz.json, as Power BI will always load the latest version from AppSource if the guid is known. By severing the guid, you also will not be able to leverage the benefits of the visual being certified (if that's what you need), as it's tied to the guid in AppSource so that people cannot abuse this method. This is how the standalone/alpha/beta builds work - the build process replaces the guid in the source code with an independent one.

[dm-p]

MS are not going to help with making dynamic properties a reality, and I've been thinking about it in conjunction with some of the other requests that have come through around better customization of the displayed elements for params/signals. I've also done research into how external element binding works with Vega, and while we can do this inside the visual with a form element that we create there, we cannot bind to elements in the properties pane, as they are outside the sandbox (and therefore invisible to a custom visual).

I would like to propose an alternative solution. We will need to do a proof-of-concept to explore how well it works, but hopefully it will be possible, and can be a good compromise for building templates that can be modified without going into the editor. Instead of relying on the Power BI formatting properties (that we will not be able to dynamically create), we:

• Provide a UI inside Deneb's editor for creating properties that we want to expose user-editable components for. This would likely be a new pane next to Config in the toolbar, meaning that the designer occupies the same space as the JSON editor.
• The person developing the spec configures the property name and type of control that they wish to use (for the PoC we will just have text and number types; if we can prove the concept, we will open this up to other types).
• We can rearrange the order that properties are displayed by dragging them (we'll add something more accessible than dragging, too).
• Anything created here will show the desired component in-line that the developer can use to specify a value and see the effect on the rendered visual.
• For properties created in this UI, Deneb will patch them as signals into the Vega view at parse and render time, providing suitable bind configuration to the components we create to allow modification of the properties.
• For properties created in this UI, Deneb will persist their values into a special property that tracks their configuration and state (much like how the spec and config are persisted).
• Deneb will handle the deserialization of this property and the re-generation of the UI for subsequent edits by the developer when the visual is reinitialized.
• This configuration (and property values) will be part of the exported template, so that it can be imported into new visuals in a similar way. Property values that are stored in the template will be the default value for visuals created with this template. if the user wishes to change the default values, they can go into the editor and modify them using the same approaches as the original developer.
• For viewers solely importing the template to work with the resulting visual and not edit it, we will create a popover component on the visual surface.
  • This will contain just the configured components and will allow them to change values from the visual surface.
  • As values are changed, they will be persisted to the visual, much like if they were changed in the editor scenario above.
  • This means that if someone is editing the report, these will be persisted like regular properties and will remain as these values when the report is saved and reopened.
  • In view mode (i.e., in the service), these properties can be modified and will be 'sticky' until the report is closed and re-opened (as settings cannot be persisted back to a report in view mode, but they do remain durable for the current report session). This would also happen with any standard Power BI-based property that we would expose to the end-user.

If a PoC works, we can consider things like:

• Adding more input types (e.g., range, radio, dropdown, date, checkbox).
• Adding groups, to allow logical grouping of properties in separate menus.
• Specifying whether properties should be visible for viewers, or just available when editing.
• Adding expression support for Vega/Vega-Lite (to make setting of initial values more dynamic).

In terms of timings, I would think that we could do a PoC and add this is a 'preview feature', which would be enabled via a property in the properties pane. If this is enabled, then visuals that have this flag enabled would provide the above functionality and allow importing of templates created with this feature. If and when we move the PoC to an MVP for all users, this feature flag would be removed and would be a supported feature.

Does this sound like a good compromise? It will be a lot of work from my end to R&D a PoC so I'd like to be sure that this could be conceptually workable for you as an option before I proceed.

[avatorl]

@dm-p I've missed to answer to you last comment somehow (well, I was badly sick back in September).

That sounds like a great workaround. Requires a few more clicks to open (vs Power BI Format pane), but sounds really good as a workaround. It would be good to have an option to make this new Format pane open by default (instead of Specification pane).

[dm-p]

I actually have a PoC put together for this but it is a little rough around the edges. I'm a bit busy with life over the next few weeks and taking some time off, but remind me sometime after 18th March (when I'm due back) and I can show you what I have as an idea so far

[HenrikJeppsson]

The main reason for this feature request is probably that vega json is "scary" for end users. And implementation of a nice looking gui solution to bind properties may be a lot of work.

I think you can get a long way with only a simple separate config page (json, yaml or simply key-value pairs of parameters). Then you could utilize magic tokens in the vega code that is replaced like a pre processor. This way folks that are fluent in vega can do the complex stuff and only expose simple parameters like BGColor.