[🐛 BUG] Filter on scenario properties in Scenario Selector not working

FlorianJacta commented 3 months ago

What went wrong? 🤔

Using custom filters on scenario properties will filter all scenarios even if the scenario respects the filters.

This issue could be irrelevant depending on what we want tot do in: https://github.com/Avaiga/taipy/issues/1511

Expected Behavior

We should filter on properties and scenarios that respect the filters should appear in the scenario selector.

Steps to Reproduce Issue

This comes from an actual use. There are three custom filters added to the scenario selector that corresponds to three properties added to the scenario.

These are the properties:

    scenario.properties['Fix warehouses'] = True
    scenario.properties['Nb of warehouses'] = 5
    scenario.properties['Fixed warehouses'] = ['Louisville']

Run the code below, try to filter on the custom filters.

# Import necessary libraries
import pandas as pd
import taipy as tp
from taipy import Config, Scope, Frequency

import datetime as dt

# Function to run a Dataiku scenario
def run_something(input_1, input_2):
    datetime = dt.datetime.now()
    date = dt.date(2018, 1, 1)
    int_var = 10
    string_var = "String"
    return datetime, date, int_var, string_var

data = {"toto": [i for i in range(10_000)], 
        "titi": [2*i for i in range(10_000)],
        "tata": [4*i for i in range(10_000)]}

input_1_cfg = Config.configure_data_node(
    id="input_1_data_node",
    default_data=data,
)

input_2_cfg = Config.configure_data_node(
    id="input_2_data_node",
    default_data=data,
)

datetime_cfg = Config.configure_data_node(id="datetime_data_node")
date_cfg = Config.configure_data_node(id="date_data_node")
int_cfg = Config.configure_data_node(id="int_data_node")
string_cfg = Config.configure_data_node(id="string_data_node")

# Scenario and task configuration in Taipy
scenario_task_cfg = Config.configure_task(
    id="scenario_task",
    function=run_something,
    input=[input_1_cfg, input_2_cfg],
    output=[datetime_cfg, date_cfg, int_cfg, string_cfg]
)

scenario_cfg = Config.configure_scenario(
    id="scenario",
    task_configs=[scenario_task_cfg],
    frequency=Frequency.DAILY)

filter_for_scenario = ["Config id",
                        "Label",
                        "Creation date", 
                        "Cycle label",
                        "Cycle start",
                        "Cycle end",
                        "Primary", 
                        "Tags", 
                        "Number of Warehouses",
                        "Fix warehouses", 
                        "Fixed warehouses"]

# GUI Markdown content
scenario_md = """
<|{scenario}|scenario_selector|filter={filter_for_scenario}|>

<|{scenario}|scenario|>
"""

def add_properties(scenario):
    scenario.properties['Fix warehouses'] = True
    scenario.properties['Number of Warehouses'] = 5
    scenario.properties['Fixed warehouses'] = ['Louisville']
    return scenario

# Main execution block with GUI setup
if __name__ == "__main__":
    tp.Core().run()
    scenario = tp.create_scenario(scenario_cfg)
    scenario = add_properties(scenario)

    tp.Gui(scenario_md).run(title="Bug replication", port=3248)

Browsers

Chrome

OS

Windows

Version of Taipy

develop - 7/17/24

Acceptance Criteria

[ ] Ensure new code is unit tested, and check code coverage is at least 90%.
[ ] Create related issue in taipy-doc for documentation and Release Notes.

Code of Conduct

[X] I have checked the existing issues.
[ ] I am willing to work on this issue (optional)

FredLL-Avaiga commented 3 months ago

Properties need to be valid python identifier And we only support string custom properties

Can you live with this @FlorianJacta ?

FlorianJacta commented 3 months ago

I can't make it work. Can you?

FredLL-Avaiga commented 3 months ago

yes with valid python identifier (ie no space) and comparing string values

I will bring an enhancement for supporting more types but the identifier request will stay (with a precise warning) I might even replace _ by space in the GUI

FredLL-Avaiga commented 3 months ago

property names have to be valid python identifier (ie no space), that's a core requirement to access it via .
even if the type of a dynamic property cannot be determined before the filter, we now try to adapt the filter value to the value read from the property at the moment of filtering
if the filter action is contains and the property value is an iterable, we try to find the value in the iterable (after adapting it to the list - element type)
string comparison are done in lower case
date comparison are done in iso string format

This applies to datanode filter as well

Avaiga / taipy