[🐛 BUG] Never receive status failed for scenario submission

FlorianJacta commented 3 months ago

What went wrong? 🤔

My scenario raises an exception, but I don't see it failing when using the scenario viewer. The status in the scenario viewer says that it is still running.

If I use the on_submission_change callback, I also don't receive the event that the scenario submission has failed.

Expected Behavior

I should receive a "fail" event for my submission.

Steps to Reproduce Issue

Run this code and submit a scenario through the UI.

notifications say it is pending and then running (but never failed).
the status of the scenario viewer says that the submission is pending and then running (but never failed).

# Import necessary libraries
import pandas as pd
import taipy as tp
from taipy import Config, Scope, Frequency
from taipy.gui import notify
import time

import datetime as dt

Config.configure_job_executions(mode="standalone", max_nb_of_workers=2)

# Function to run a Dataiku scenario
def run_something(input_1, input_2):
    datetime = dt.datetime.now()
    date = dt.date(2018, 1, 1)
    int_var = 10
    string_var = "String"
    raise(Exception("Something totally unexpected happened"))
    return datetime, date, int_var, string_var

data = {"toto": [i for i in range(10_000)], 
        "titi": [2*i for i in range(10_000)],
        "tata": [4*i for i in range(10_000)]}

input_1_cfg = Config.configure_data_node(
    id="input_1_data_node",
    default_data=data,
)

input_2_cfg = Config.configure_data_node(
    id="input_2_data_node",
    default_data=data,
)

datetime_cfg = Config.configure_data_node(id="datetime_data_node")
date_cfg = Config.configure_data_node(id="date_data_node")
int_cfg = Config.configure_data_node(id="int_data_node")
string_cfg = Config.configure_data_node(id="string_data_node")

# Scenario and task configuration in Taipy
scenario_task_cfg = Config.configure_task(
    id="scenario_task",
    function=run_something,
    input=[input_1_cfg, input_2_cfg],
    output=[datetime_cfg, date_cfg, int_cfg, string_cfg]
)

scenario_cfg = Config.configure_scenario(
    id="scenario",
    task_configs=[scenario_task_cfg],
    frequency=Frequency.DAILY)

# GUI Markdown content
scenario_md = """
<|{scenario}|scenario_selector|>

<|{scenario}|scenario|on_submission_change=track_scenario_change|>
"""

def track_scenario_change(state, submittable, details):
    print("Track scenario change", submittable, details)
    notify(state, "info", f"Track scenario change -> {details['submission_status']}")

# Main execution block with GUI setup
if __name__ == "__main__":
    tp.Core().run()
    scenario = tp.create_scenario(scenario_cfg)
    scenario = tp.create_scenario(scenario_cfg)

    tp.Gui(scenario_md).run(title="Bug replication", port=3248)

Version of Taipy

develop

Acceptance Criteria

[ ] Ensure new code is unit tested, and check code coverage is at least 90%.
[ ] Create related issue in taipy-doc for documentation and Release Notes.

Code of Conduct

[X] I have checked the existing issues.
[ ] I am willing to work on this issue (optional)

FredLL-Avaiga commented 2 months ago

I've done some exploring and logging:

I do see the scenario and can submit it.

I see an exception in the python console but nothing in the UI (ie submission stays pending)

Exception: Something totally unexpected happened
[2024-08-28 11:16:18.232][Taipy][WARNING] JOB_scenario_task_99110940-e0f1-46b1-b377-8e08938e65fd is not in the blocked list anymore.
[2024-08-28 11:16:18.303][Taipy][ERROR]  1 errors occurred during execution of job JOB_scenario_task_99110940-e0f1-46b1-b377-8e08938e65fd
[2024-08-28 11:16:18.304][Taipy][ERROR] Exception: Something totally unexpected happened

FredLL-Avaiga commented 2 months ago

If I add a log of each received event

Exception: Something totally unexpected happened
Event Event(entity_type=<EventEntityType.JOB: 6>, operation=<EventOperation.UPDATE: 2>, entity_id='JOB_scenario_task_d60deb09-901b-45a7-bce4-6d59884077d5', attribute_name='status', attribute_value=<Status.FAILED: 6>, metadata={'creation_date': datetime.datetime(2024, 8, 28, 11, 20, 20, 875101), 'task_config_id': 'scenario_task', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 18009))
[2024-08-28 11:20:27.236][Taipy][WARNING] JOB_scenario_task_d60deb09-901b-45a7-bce4-6d59884077d5 is not in the blocked list anymore.
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_datetime_data_node_ff623edd-221e-4cc5-8879-511f29aab159', attribute_name='editor_id', attribute_value=None, metadata={'config_id': 'datetime_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 241947))        
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_datetime_data_node_ff623edd-221e-4cc5-8879-511f29aab159', attribute_name='editor_expiration_date', attribute_value=None, metadata={'config_id': 'datetime_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 243958))
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_datetime_data_node_ff623edd-221e-4cc5-8879-511f29aab159', attribute_name='edit_in_progress', attribute_value=False, metadata={'config_id': 'datetime_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 246957))
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_date_data_node_48205254-ccc1-494c-90b8-1cff191fe214', attribute_name='editor_id', attribute_value=None, metadata={'config_id': 'date_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 260488))
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_date_data_node_48205254-ccc1-494c-90b8-1cff191fe214', attribute_name='editor_expiration_date', attribute_value=None, metadata={'config_id': 'date_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 262173))   
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_date_data_node_48205254-ccc1-494c-90b8-1cff191fe214', attribute_name='edit_in_progress', attribute_value=False, metadata={'config_id': 'date_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 264011))        
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_int_data_node_687b7e9b-c553-4b07-a303-70c0bbeb9f4a', attribute_name='editor_id', attribute_value=None, metadata={'config_id': 'int_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 274534))
[2024-08-28 11:20:27.308][Taipy][ERROR]  1 errors occurred during execution of job JOB_scenario_task_d60deb09-901b-45a7-bce4-6d59884077d5
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_int_data_node_687b7e9b-c553-4b07-a303-70c0bbeb9f4a', attribute_name='editor_expiration_date', attribute_value=None, metadata={'config_id': 'int_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 276533))     
[2024-08-28 11:20:27.309][Taipy][ERROR] Exception: Something totally unexpected happened
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_int_data_node_687b7e9b-c553-4b07-a303-70c0bbeb9f4a', attribute_name='edit_in_progress', attribute_value=False, metadata={'config_id': 'int_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 279798))
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_string_data_node_6dacd1d7-4673-442e-a5e3-18c881783648', attribute_name='editor_id', attribute_value=None, metadata={'config_id': 'string_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 292846))
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_string_data_node_6dacd1d7-4673-442e-a5e3-18c881783648', attribute_name='editor_expiration_date', attribute_value=None, metadata={'config_id': 'string_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 294852))
Event Event(entity_type=<EventEntityType.DATA_NODE: 5>, operation=<EventOperation.UPDATE: 2>, entity_id='DATANODE_string_data_node_6dacd1d7-4673-442e-a5e3-18c881783648', attribute_name='edit_in_progress', attribute_value=False, metadata={'config_id': 'string_data_node', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 297851))    
Event Event(entity_type=<EventEntityType.JOB: 6>, operation=<EventOperation.UPDATE: 2>, entity_id='JOB_scenario_task_d60deb09-901b-45a7-bce4-6d59884077d5', attribute_name='execution_ended_at', attribute_value=datetime.datetime(2024, 8, 28, 11, 20, 27, 310375), metadata={'creation_date': datetime.datetime(2024, 8, 28, 11, 20, 20, 875101), 'task_config_id': 'scenario_task', 'version': '0880e210-eb76-490c-ba8e-09488dc49586'}, creation_date=datetime.datetime(2024, 8, 28, 11, 20, 27, 310375))

FredLL-Avaiga commented 2 months ago

there is an update on the status of the job which goes to FAILED but there's no update for the submission which is used to update the scenario UI @toan-quach Do you think an event for Submission could/should be sent ?

toan-quach commented 2 months ago

I have fixed the issue of not publishing the "FAILED" submission status https://github.com/Avaiga/taipy/pull/1722

However, I noticed that the status of the scenario is not updated :/

FredLL-Avaiga commented 2 months ago

tx @toan-quach let me know when your PR is merged and I'll have another look

FredLL-Avaiga commented 2 months ago

@toan-quach I've tested with your branch: works for me (tm)

toan-quach commented 2 months ago

awesome thanks @FredLL-Avaiga !

github-actions[bot] commented 1 month ago

This issue has been labelled as "🥶Waiting for contributor" because it has been inactive for more than 14 days. If you would like to continue working on this issue, please add another comment or create a PR that links to this issue. If a PR has already been created which refers to this issue, then you should explicitly mention this issue in the relevant PR. Otherwise, you will be unassigned in 14 days. For more information please refer to the contributing guidelines.

jrobinAV commented 1 month ago

@toan-quach @FredLL-Avaiga @FlorianJacta I believe this issue should be closed. Am I correct?

FredLL-Avaiga commented 1 month ago

always happy to close

Avaiga / taipy