Make complex filter behavior simple

[FlorianJacta]

Description

I have an example coming from a real life use case from a customer about filtering a table.

The behavior is as followed (very similar to the Excel behavior for filters):

• I have two multiple filters filtering my table,

• Their default value is "All",
• If "All" is selected then the table is not filtered based on this filter,
• If another value is selected then the table is filtered based on this value AND "All" is removed automatically from the list of selected values,

• The list of values (lov) of the other filters are changed according to it (if you filter on "Paris" then the lov of "facility_type" is ["All", "Small", "Medium"] because Paris only has "Small" and "Medium" facilities and not "Large" facilities.

• If they are no selected values then the selected_value becomes ["All"]

Solution Proposed

Here is the code created for this demo. The goal is to simplify this code and make this behavior or part of it inside Taipy directly.

import taipy.gui.builder as tgb
import pandas as pd
from taipy import Gui

data = pd.DataFrame(
    {
        "city": [
            "Paris",
            "Paris",
            "London",
            "London",
            "London",
            "New York",
            "New York",
            "New York",
            "New York",
            "New York",
            "Sartrouville",
        ],
        "facility_type": [
            "Small",
            "Medium",
            "Large",
            "Small",
            "Medium",
            "Large",
            "Small",
            "Medium",
            "Large",
            "Small",
            "Medium",
        ],
        "value": [0, 31, 5, 1, 2, 30, 40, 36, 134, 45.2, 5.6224],
    }
)
displayed_data = data.copy()

selected_facility_type = previous_selected_facility_type = ["All"]
selected_city = previous_selected_city = ["All"]

lov_selected_facility_type = ["All"] + data["facility_type"].unique().tolist()
lov_selected_city = ["All"] + data["city"].unique().tolist()

list_of_lov = [
    (
        "lov_selected_facility_type",
        "facility_type",
        "selected_facility_type",
        lov_selected_facility_type,
    ),
    ("lov_selected_city", "city", "selected_city", lov_selected_city),
]

def filter_displayed_data(state, var_name, var_value):
    def handle_selected_value(state, var_name, var_value):
        if var_name:
            new_selected_values = getattr(state, var_name)
            if len(new_selected_values) > 1 and "All" in getattr(
                state, f"previous_{var_name}"
            ):
                new_value = [
                    string for string in new_selected_values if string != "All"
                ]
                setattr(state, var_name, new_value)
                setattr(state, "previous_" + var_name, new_value)
            elif (
                "All" in new_selected_values
                and "All" not in getattr(state, "previous_" + var_name)
            ) or (len(new_selected_values) == 0):
                new_value = ["All"]
                setattr(state, var_name, new_value)
                setattr(state, "previous_" + var_name, new_value)

    def handle_lov(state, var_name, var_value):
        if var_name and var_name:
            for filter_name in list_of_lov:
                if filter_name[0] != f"lov_{var_name}" and "All" in getattr(
                    state, filter_name[2]
                ):
                    new_lov = ["All"] + filtered_data[filter_name[1]].astype(
                        str
                    ).unique().tolist()
                    setattr(state, filter_name[0], new_lov)

    handle_selected_value(state, var_name, var_value)

    # Start with the unfiltered data
    filtered_data = state.data.copy()

    filters_multiple = {
        "city": state.selected_city,
        "facility_type": state.selected_facility_type,
    }

    for column, selected_values in filters_multiple.items():
        if "All" not in selected_values:
            filtered_data = filtered_data[filtered_data[column].isin(selected_values)]

    # Apply the date and price range filters
    filtered_data.reset_index(drop=True, inplace=True)

    handle_lov(state, var_name, var_value)

    filtered_data = filtered_data.round(2)

    # Update the displayed data in the state
    state.displayed_data = filtered_data

with tgb.Page() as page:
    # Filter for facility type
    tgb.selector(
        value="{selected_facility_type}",
        lov="{lov_selected_facility_type}",
        dropdown=True,
        filter=True,
        label="Facility type",
        on_change=filter_displayed_data,
        class_name="fullwidth m-half",
        multiple=True,
    )

    # Filter for city
    tgb.selector(
        value="{selected_city}",
        lov="{lov_selected_city}",
        dropdown=True,
        filter=True,
        label="City",
        on_change=filter_displayed_data,
        class_name="fullwidth m-half",
        multiple=True,
    )

    tgb.table("{displayed_data}")

Gui(page).run()

Acceptance Criteria

• [ ] If applicable, a new demo code is provided to show the new feature in action.
• [ ] Integration tests exhibiting how the functionality works are added.
• [ ] Any new code is covered by a unit tested.
• [ ] Check code coverage is at least 90%.
• [ ] Related issue(s) in taipy-doc are created for documentation and Release Notes are updated.

Code of Conduct

• [X] I have checked the existing issues.
• [ ] I am willing to work on this issue (optional)

[FredLL-Avaiga]

this is really use-case specific

[FlorianJacta]

This is really specific but it happened for two clients already.

I think the "All" value is not generic and is extremely useful.

[jrobinAV]

In terms of UX, I would use something like GitHub. Instead of having a specific "All" entry acting like an exception and potentially conflicting with an actual "All" value, I would add an icon/button to remove all filters. Just like the following screenshot:

[FlorianJacta]

I agree; the "All" value is here because I have to create a workaround. The firsr client wanted a behavior similar to Excel. I don't think we have to copy exactly what Excel has done because their drop-down selectors changes depending on the table and the filters