[OTHER] High latency between scenario submission to job pending

[noobHappylife]

What would you like to share or ask?

Tested on Taipy v3.1.1.

I've experienced something strange, on a ubuntu server, using scenario viz element, between submit (button clicked) to first log of submission status change to pending can be slow, up-to 10 or 20 seconds. However, on a local PC it's usually very fast.

After upgrade to Taipy v4.0.0dev. It works much better, close to what I've experienced on a local PC.

Code of Conduct

• [X] I have checked the existing issues.
• [ ] I am willing to work on this issue (optional)

[Lymah123]

Hi @jrobinAV, Can you tell me more about this issue?

[AkashJana18]

I would like to work on this issue as hacktober participant can you assign me this issue? Also please guide how to proceed with issue as the description of the issue seems abstract

[FlorianJacta]

@AkashJana18 You have been assigned to another issue.

@jrobinAV Do you have more information to add to this issue for @Lymah123 ?

[quest-bot[bot]]

New Quest!

A new Quest has been launched in @Avaiga’s repo. Merge a PR that solves this issue to loot the Quest and earn your reward.

---

Some loot has been stashed in this issue to reward the solver!

🗡 Comment @quest-bot embark to check-in for this Quest and start solving the issue. Other solvers will be notified!

⚔️ When you submit a PR, comment @quest-bot loot #1714 to link your PR to this Quest.

Questions? Check out the docs.

[xyfer17]

@FlorianJacta @noobHappylife Can you assign this issue to me, if no one is working on it.

[FlorianJacta]

@AkashJana18 The user has experienced a performance issue when the code was deployed on a ubuntu server. This issue happened on the Ubuntu server and with Taipy 3.1.1.

You could try this code maybe:

from taipy import Config
import taipy as tp
import pandas as pd
import datetime as dt

data = pd.read_csv(
    "https://raw.githubusercontent.com/Avaiga/taipy-getting-started-core/develop/src/daily-min-temperatures.csv"
)

# Normal function used by Taipy
def predict(
    historical_temperature: pd.DataFrame, date_to_forecast: dt.datetime
) -> float:
    print(f"Running baseline...")
    historical_temperature["Date"] = pd.to_datetime(historical_temperature["Date"])
    historical_same_day = historical_temperature.loc[
        (historical_temperature["Date"].dt.day == date_to_forecast.day)
        & (historical_temperature["Date"].dt.month == date_to_forecast.month)
    ]
    return historical_same_day["Temp"].mean()

# Configuration of Data Nodes
historical_temperature_cfg = Config.configure_data_node("historical_temperature")
date_to_forecast_cfg = Config.configure_data_node("date_to_forecast")
predictions_cfg = Config.configure_data_node("predictions")

# Configuration of tasks
predictions_cfg = Config.configure_task(
    "predict",
    predict,
    [historical_temperature_cfg, date_to_forecast_cfg],
    predictions_cfg,
)

# Configuration of scenario
scenario_cfg = Config.configure_scenario(
    id="my_scenario", task_configs=[predictions_cfg]
)

Config.export("config.toml")

if __name__ == "__main__":
    # Run of the Core
    tp.Core().run()

    # Creation of the scenario and execution
    scenario = tp.create_scenario(scenario_cfg)
    scenario.historical_temperature.write(data)
    scenario.date_to_forecast.write(dt.datetime.now())
    tp.submit(scenario)

    print("Value at the end of task", scenario.predictions.read())

    def save(state):
        state.scenario.historical_temperature.write(data)
        state.scenario.date_to_forecast.write(state.date)
        state.refresh("scenario")
        tp.gui.notify(state, "s", "Saved! Ready to submit")

    date = None
    scenario_md = """
<|{scenario}|scenario_selector|>

Select a Date
<|{date}|date|on_change=save|active={scenario}|>

Run the scenario
<|{scenario}|scenario|>
<|{scenario}|scenario_dag|>

View all the information on your prediction here
<|{scenario.predictions}|data_node|>
"""

    tp.Gui(scenario_md).run()

[AkashJana18]

@FlorianJacta I have already been assigned to another to it would be better if anyone else take this issue as it also has the hacktober tag on this... if no one is willing to work on this then i will try to solve it after I finish the other issue

[FlorianJacta]

We have changed our policy so that anyone can take any issues, whether assigned or not. If someone else wants to be assigned to this issue, he will be able to.

[Lymah123]

I have started working on the issue already

[jrobinAV]

@Lymah123 Sorry for the late answer. I don't have more information than what is on the issue:

• deployed on an ubuntu server
• version 3.1.1 of taipy

Maybe @noobHappylife got more info. Do you run the Core service in standalone mode (The job_config)? Any deployment details like the machine configuration? Any log or piece of code to reproduce the issue?

[noobHappylife]

Hi, Here are some more information:

• was running in standalone mode

  Config.configure_job_executions(mode="standalone", max_nb_of_workers=1)

• initially worked on windows 10 machine, with taipy version 3.1.1. The delay between clicking scenario submit button in the UI and the task starts to run is very low (basically instant reaction).
• when i moved to a ubuntu server 20.04, with the same code. The delay between clicking scenario submit button in the UI and the task starts to run fluctuate hugely, and can be very slow up to tens of seconds.
• I tried using my own code, and just a simple straight forward scenario example from the gallery and it behaves similarly.
• After I upgrade to taipy version 4.0.0dev0 on ubuntu server the delay becomes much more consistent, and closer to what I experienced on windows machine.

Hope it helps.

[Lymah123]

@noobHappylife , thank you.

[github-actions[bot]]

This issue has been labelled as "🥶Waiting for contributor" because it has been inactive for more than 14 days. If you would like to continue working on this issue, please add another comment or create a PR that links to this issue. If a PR has already been created which refers to this issue, then you should explicitly mention this issue in the relevant PR. Otherwise, you will be unassigned in 14 days. For more information please refer to the contributing guidelines.

[github-actions[bot]]

This issue has been unassigned automatically because it has been marked as "🥶Waiting for contributor" for more than 14 days with no activity.