Avaiga / taipy-core

A Python library to build powerful and customized data-driven back-end applications.
Apache License 2.0
38 stars 17 forks source link

Problem of version compability even with Develop mode on #743

Closed FlorianJacta closed 1 year ago

FlorianJacta commented 1 year ago

Description

Changing configuration might create issues in Taipy Core even in Develop mode. Changing the name of the function used by a task raises an error.

How to reproduce

from taipy import Config, Scope, Frequency

import datetime as dt

import taipy as tp
import pandas as pd

PATH_CSV = 'dataset.csv'
PATH_PARQUET = 'dataset.parquet'

def transform(csv, parquet, pickle):
    print("     Cleaning data")
    return csv, 5, "hello", dt.datetime.now()

## Input Data csv
csv_cfg = Config.configure_data_node(id="csv",  storage_type="csv", path=PATH_CSV)#scope=Scope.GLOBAL,
parquet_cfg = Config.configure_data_node(id="parquet", scope=Scope.CYCLE, storage_type="parquet", path=PATH_PARQUET)
pickle_cfg = Config.configure_data_node(id="pickle")

## Remaining Data Node
data_out_cfg = Config.configure_data_node(id="data_out")
int_cfg = Config.configure_data_node(id="int")
string_cfg = Config.configure_data_node(id="string")
date_cfg = Config.configure_data_node(id="date")

# Task config objects
transform_task_cfg = Config.configure_task(id="transform",
                                            function=transform,
                                            input=[csv_cfg,parquet_cfg,pickle_cfg],
                                            output=[data_out_cfg, int_cfg, string_cfg, date_cfg],
                                            skippable=True)

# Configure our scenario config.
scenario_cfg = Config.configure_scenario(id="scenario", task_configs=[transform_task_cfg], frequency=Frequency.MONTHLY)

if __name__ == "__main__":
    tp.Core().run()

    scenario = tp.create_scenario(config=scenario_cfg)
    data_pickle = pd.DataFrame({"Hello": [1, 2, 3], "World": [4, 5, 6]})
    scenario.pickle.write(data_pickle)
    data_csv = pd.DataFrame({"Hi": ["red", "step", 'true'], "World": [None, 5, 6]})
    data_csv.to_csv(PATH_CSV)
    data_parquet = pd.DataFrame({"Date":[dt.datetime(2021, 1, 1), dt.datetime(2021, 1, 2), dt.datetime(2021, 1, 3)], "Value": [1, 2, 3]})
    data_parquet.to_parquet(PATH_PARQUET)

    print(scenario.pickle.read())
from taipy import Config, Scope, Frequency

import datetime as dt

import taipy as tp
import pandas as pd

PATH_CSV = 'dataset.csv'
PATH_PARQUET = 'dataset.parquet'

def other_function(csv, parquet, pickle):
    print("     Cleaning data")
    return csv, 5, "hello", dt.datetime.now()

## Input Data csv
csv_cfg = Config.configure_data_node(id="csv",  storage_type="csv", path=PATH_CSV)#scope=Scope.GLOBAL,
parquet_cfg = Config.configure_data_node(id="parquet", scope=Scope.CYCLE, storage_type="parquet", path=PATH_PARQUET)
pickle_cfg = Config.configure_data_node(id="pickle")

## Remaining Data Node
data_out_cfg = Config.configure_data_node(id="data_out")
int_cfg = Config.configure_data_node(id="int")
string_cfg = Config.configure_data_node(id="string")
date_cfg = Config.configure_data_node(id="date")

# Task config objects
transform_task_cfg = Config.configure_task(id="transform",
                                            function=other_function,
                                            input=[csv_cfg,parquet_cfg,pickle_cfg],
                                            output=[data_out_cfg, int_cfg, string_cfg, date_cfg],
                                            skippable=True)

# Configure our scenario config.
scenario_cfg = Config.configure_scenario(id="scenario", task_configs=[transform_task_cfg], frequency=Frequency.MONTHLY)

if __name__ == "__main__":
    tp.Core().run()

    scenario = tp.create_scenario(config=scenario_cfg)
    data_pickle = pd.DataFrame({"Hello": [1, 2, 3], "World": [4, 5, 6]})
    scenario.pickle.write(data_pickle)
    data_csv = pd.DataFrame({"Hi": ["red", "step", 'true'], "World": [None, 5, 6]})
    data_csv.to_csv(PATH_CSV)
    data_parquet = pd.DataFrame({"Date":[dt.datetime(2021, 1, 1), dt.datetime(2021, 1, 2), dt.datetime(2021, 1, 3)], "Value": [1, 2, 3]})
    data_parquet.to_parquet(PATH_PARQUET)

    print(scenario.pickle.read())

Expected behavior This shouldn't create any issue as we are in develop mode for Taipy Core.

Screenshots When available and relevant, screenshots better help show the problem.

Runtime environment Taipy 3.0 develop

jrobinAV commented 1 year ago

Thank you @FlorianJacta. We will have a look at it. Probably the next sprint.