[Tidy] Proof of concept replacing ctd for filter and parameter

antonymilne commented 1 week ago

Description

@petar-qb raising this PR just to see if you think it's a good idea as a change. I won't actually implement it until you've done your PR so you don't have to resolve any more conflicts.

I'm finally continuing work on #363 and this seems like a small refactor that will tidy things up some more before the conversion to classes, which will hopefully be the next PR I raise on this topic 🤞

So far this PR is just a proof of concept to show what the change would look like. I haven't fully rolled it out or updated tests yet but the simple demo app works exactly the same as before.

Basically we passed filters and parameters through inputs in the callbacks before but didn't actually use them anywhere and instead looked inside ctx.args_grouping to extract all the values and and component ids. The only actual properties used in the CallbackTriggerDict are id and value. We can instead pass through named states in the form {<id>: State(...)} to get the id passed in automatically so we don't need ctx.args_grouping for that. I don't think we'll ever need it for anything else. Here's what's in it to remind you:

class CallbackTriggerDict(TypedDict):
    """Represent dash.ctx.args_grouping item. Shortened as 'ctd' in the code.

    Args:
        id: The component ID. If it's a pattern matching ID, it will be a dict.
        property: The component property used in the callback.
        value: The value of the component property at the time the callback was fired.
        str_id: For pattern matching IDs, it's the stringified dict ID without white spaces.
        triggered: A boolean indicating whether this input triggered the callback.

    """

    id: ModelID
    property: Literal["clickData", "value", "n_clicks", "active_cell", "derived_viewport_data"]
    value: Optional[Any]
    str_id: str
    triggered: bool

The change here would achieve be half of what you suggest in this comment:

        # TODO-AV2-OQ: Consider the following inputs ctx form:
        #  ```
        #  return {
        #      target_1: {'filters': ..., 'parameters': ..., 'filter_interaction': ..., 'theme_selector': ...},
        #      target_2: {'filters': ..., 'parameters': ..., 'filter_interaction': ..., 'theme_selector': ...},
        #  }
        #  ```
        #  Pros:
        #  1. We don't need anymore to send all filter/parameters/filter_interaction inputs to the server
        #  2. Potentially we don't need to dig through the ctx in the _actions_utils.py which decreases the complexity

The other half is also captured by my more recent comment:

    # TODO: the structure here would be nicer if we could get just the ctds for a single target at one time,
    #  so you could do apply_filters on a target a pass only the ctds relevant for that target.
    #  Consider restructuring ctds to a more convenient form to make this possible.

Screenshot

Notice

[ ] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
- I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
- I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
- I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
- I have not referenced individuals, products or companies in any commits, directly or indirectly.
- I have not added data or restricted code in any commits, directly or indirectly.

github-actions[bot] commented 1 week ago

View the example dashboards of the current commit live on PyCafe :coffee: :rocket:

Updated on: 2024-11-15 15:26:40 UTC Commit: 94a90cdff5050a99cfac60a5e0d0f7561fa0b5b2

Link: vizro-core/examples/dev/

Link: vizro-core/examples/scratch_dev

Link: vizro-core/examples/visual-vocabulary/

Link: vizro-ai/examples/dashboard_ui/

petar-qb commented 1 week ago

I like the initiative! There's a huge room for improvement regarding the ctds. We have to find the best possible format for sending these from the client to the server.

In your code version, states are sent like:

dict(component_id_1=State(componenent_id, ...), component_id_2=...).

It's similar to the current filter_interaction format. However, for the filter_interaction, we decided to sent it like this

list({
        "clickData": State(component_id=self.id, component_property="clickData"),
        "modelID": State(component_id=self.id, component_property="id"),  # required, to determine triggered model
    }, 
    {...},
)

The current filter_interaction way enables to propagate multiple different properties from of the same component to the server. So, we should preserve this flexibility I think.

N.B. There are two tickets we should keep in mind while thinking about the ctds changes:

antonymilne commented 3 days ago

Great point, thanks @petar-qb. You're right that my proposed structure doesn't cater for one component with multiple States.

After thinking about this more, I'm going to close this PR for now but open a ticket so we can revisit another time.

Ultimately I'd like to try and handle controls using pattern-matching callback like this: https://github.com/plotly/dash/issues/3063

@callback(
    Output("graph", "figure"), 
    State({"control_type": "filter", "id": ALL}, "id"), 
    State({"control_type": "filter", "id": ALL}, "value")
)

and then do zip(ids, property_values) to match together the component id and value. When we have controls inside containers imagine you'd address State({"control_type": "filter", "container": "container_name", "id": ALL}, "id") or something. This would mean that we can remove a lot of the code that basically handles "find all the controls on this page".

There's only two possible drawbacks with this I think:

it relies on Dash guaranteeing the order of ALL inputs - see https://github.com/plotly/dash/issues/3063. Hopefully that works in future
we still need to be able to use ids for CSS selectors and other callbacks easily (see 1 2. Assuming we do pattern matching just for controls (I don't see any use for it elsewhere anyway) we need the dictionary id to be associated with the selector itself. So we could still put the human-readable filter/parameter model id in some outer wrapper of a selector. We'd potentially need a translation to convert between someone targetting a target=["filter_id"] and the more complicated selector id. Hopefully we can avoid that by just putting ids in the right places and building the right innermost dynamically though just like in #879

As a first step we might adopt the same structure but without pattern-matching ids like this:

ids=[State("component1", "id"), State("component1", "id"), State("component2", "id")], 
values=[State("component1", "clickData"), State("component1", "somethingElse"), State("component2", "value")]

...but I'm not in a big hurry to do that because it will be heavier changes across the codebase and is not a priority. Let's try and do this only after we've removed ctds_filter_interaction.

There's not much point doing a solution like I suggested here with {<id>: State(...)} because it won't generalise to pattern-matching in future.

Let me know if it makes sense and I'll make a ticket for it and close this PR.

petar-qb commented 2 days ago

Hey @antonymilne. Thanks for the great investigation!! Pattern-matching looks really promising, and from the top of my head it probably looks like the end goal in dealing with controls as actions inputs. (This solution looks like it could enable -> https://github.com/McK-Internal/vizro-internal/issues/576, but I can't guarantee that right now)

but I'm not in a big hurry to do that because it will be heavier changes across the codebase and is not a priority. Let's try and do this only after we've removed ctds_filter_interaction.

There's not much point doing a solution like I suggested here with {: State(...)} because it won't generalise to pattern-matching in future.

I totally agree with these two points. 👍 Let's create a ticket and deal with the ctds structure after we get rid of ctds_filter_interaction.

antonymilne commented 2 days ago

Cool, thanks @petar-qb! Indeed this could solve a lot of things. e.g. also think of cases where a control could be added to a page dynamically. I don't think this would actually be possible without pattern-matching callbacks.

I've made a new issue here: https://github.com/McK-Internal/vizro-internal/issues/1376.

In due course I'll also make an issue to track forthcoming breaking changes to expect in 0.2.0, which currently exists just in rough notes on my computer.

Once we've got the next couple of PRs on filters and actions done let's re-prioritise all the various actions v2 tickets because things have become a lot clearer to me over the last couple of months, and some tickets may no longer be relevant.

mckinsey / vizro