Cycle based filtering for scenario getters

[jrobinAV]

Propose a way to filter scenarios by providing a cycle description.

Objectives

We want to implement a filter mechanism, a query system, or Python API to describe and retrieve a meaningful subset of scenarios easily. This filtering should be based on the cycle. (thanks to @Forchapeatl Pull Request 😉, I realize that even if we want to expose an API to retrieve a subset of scenarios, the implementation is related to cycle description).

use cases:

In the following, I do not propose implementing a get_scenarios function exactly as written. It is the easiest way I found to describe what should be provided by the user and what should be returned by the filtering mechanism. These are some use cases I have in mind. We should have similar descriptions for weekly, monthly, quarterly, and yearly scenarios once we have listed all the relevant use cases. We can discuss an API for that.

Daily scenarios

Here are a few use cases I have in mind for daily scenarios.

• get_scenarios(days_of_week="week_days") Returns daily scenarios whose cycle corresponds to a weekday
• get_scenarios(days_of_week="week_end") Returns daily scenarios whose cycle corresponds to a Saturday or a Sunday
• get_scenarios(days_of_week=["Mondays", "Thursdays"]) Returns daily scenarios whose cycle corresponds to a Monday or a Thursday
• get_scenarios(days_of_month=[1, 15, 21]) Returns daily scenarios whose cycle corresponds to the 1st, the 15th, or the 21st day of a month.
• get_scenarios(days_of_month="last") Returns daily scenarios whose cycle corresponds to the last day of the month (28, 29, 30, or 31, depending on the month).
• get_scenarios(days_of_year=["1/1", "12/5", "22/8"]) Returns daily scenarios whose cycle corresponds to specific dates: Jan 1st, May 12th, or Aug 22nd.
• get_scenarios_by(days_of_year=["January", "March"]) Returns daily scenarios whose cycle corresponds to a January or March day.

Weekly scenarios

TODO

[Forchapeatl]

I am great at graph algorithims, I am very much interested in this issue. May I handle this issue ?

[trgiangdo]

Thank you @Forchapeatl. If you have any question regarding the implementation, feel free to ask us.

[jrobinAV]

Thank you very much @Forchapeatl. You rock!

[Forchapeatl]

Thank you very much @Forchapeatl. You rock!

Thank you :)

[Forchapeatl]

Case study : Get all wednesday Scenarioes

From the above illustration , if we intend to get all wednesday senarios , it sould be possible by getting all senarios who's start and end date contains wednesday ( by date ) . That is we get all scenarios whoes cycle comply to this function startDate <= query day <= end Date

i would like to get more information on this function . Please tell me more about this _build_filters_with_version("all") code.

https://github.com/Avaiga/taipy/blob/170a14b149544fdde085ba17ed19719cdf790370/taipy/core/scenario/_scenario_manager.py#L255-L262

[trgiangdo]

About the _build_filters_with_version(), because most Taipy entities have a version, this method create a filter dictionary to make sure getting the correct entities of the correct version. However, Cycle entities do not have a version (it can be shared across application versions), the filter will be empty to get all Cycle entities.

[jrobinAV]

This issue is not ready to be implemented for me. It requires a better description of the motivation, the expected output, a description of the expected API and a list of use cases.

[jrobinAV]

@Forchapeatl @toan-quach @trgiangdo @FlorianJacta I just updated the description of the issue to try to make it clearer.

Please provide any feedback, comments, or ideas so we can challenge, or append the use cases. Once we settle on representative use cases we can propose an API.

What do you think?

[Forchapeatl]

Thank you for elaborating on the usecases @jrobinAV . It 's very good. The way I see it , we could split the usecases into 2 functions.

1. Get daily scenarios by date: This method can take datetime as parameter and bool if we want it by year or month.

get_scenarios(days_of_month=[1, 15, 21]) get_scenarios(days_of_year=["1/1", "12/5", "22/8"])

2. Get daily scenarios by day : For these method the user passes an option and the function will have to compute the datetime.

get_scenarios(days_of_week="week_days") get_scenarios(days_of_week="week_end") get_scenarios(days_of_week=["Mondays", "Thursdays"]) get_scenarios(days_of_month="last") get_scenarios_by(days_of_year=["January", "March"])

@jrobinAV , please 🙏 at your convenience help draw up a few usecases for WEEKLY scenarions for me please. I am still struggling with building an organised logic together.

[trgiangdo]

We can provide multiple options for the user to filter with each type of frequency. The user can only use 1 param.

Daily scenarios

• get_scenarios(daily_cycle_on="week_days") Returns daily scenarios whose cycle corresponds to a weekday
• get_scenarios(daily_cycle_on="week_end") Returns daily scenarios whose cycle corresponds to a Saturday or a Sunday
• get_scenarios(daily_cycle_on=["Mondays", "Thursdays"]) Returns daily scenarios whose cycle corresponds to a Monday or a Thursday
• get_scenarios(daily_cycle_on=[1, 15, 21]) Returns daily scenarios whose cycle corresponds to the 1st, the 15th, or the 21st day of a month.
• get_scenarios(daily_cycle_on=-1) Returns daily scenarios whose cycle corresponds to the last day of the month (28, 29, 30, or 31, depending on the month).
• get_scenarios(daily_cycle_on=["1/1", "12/5", "22/8"]) Returns daily scenarios whose cycle corresponds to specific dates: Jan 1st, May 12th, or Aug 22nd.
• get_scenarios_by(daily_cycle_on=["January", "March"]) Returns daily scenarios whose cycle corresponds to a January or March day.
• get_scenarios_by(daily_cycle_on=[2012, 2014]) Returns daily scenarios whose cycle corresponds to a 2012 or 2014 day.

Weekly scenarios

• get_scenarios(weekly_cycle_on=["January", "March"]) Returns weekly scenarios whose cycle corresponds to a January or March week. The week should be entirely in one of the month.
• get_scenarios_by(daily_cycle_on=[2012, 2014]) Returns daily scenarios whose cycle corresponds to a 2012 or 2014 week. The week should be entirely in one of the year.

Monthly scenarios

• get_scenarios(monthly_cycle_on=["January", "March"]) Returns monthly scenarios whose cycle corresponds to a January or March.
• get_scenarios_by(monthly_cycle_on=[2012, 2014]) Returns monthly scenarios whose cycle corresponds to a 2012 or 2014 month.

Quarterly scenarios

• get_scenarios(quarterly_cycle_on=[1, 3]) Returns quarterly scenarios whose cycle corresponds to the 1st or 3rd quarter.
• get_scenarios_by(quarterly_cycle_on=[2012, 2014]) Returns quarterly scenarios whose cycle corresponds to 2012 or 2014.

Yearly scenarios

• get_scenarios_by(yearly_cycle_on=[2012, 2014]) Returns yearly scenarios whose cycle corresponds to 2012 or 2014.

[FlorianJacta]

I don't really see the relationship between cycles and this API. Does this mean that scenarios without a cycle couldn't be filtered this way?

[trgiangdo]

Yes, scenarios without cycle can be filtered using other properties (like creation_time)

[FlorianJacta]

To be honest, it is hard for me to see the motivation behind all of it. Would it be faster than if I did it myself?

What I needed in the past for cycles is a way to create a tree (as a dict) of cycles and scenarios:

• Cycle 1
  • Scenario 1
  • Scenario 2
• Cycle 2
  • Scenario 3

This was needed to create a scenario selector.

And I also needed to get all scenarios from a cycle object.

[trgiangdo]

For "a way to create a tree (as a dict) of cycles and scenarios", the API already exists

import taipy as tp
tp.get_cycles_scenarios()

For "I also needed to get all scenarios from a cycle object", the API also already exists

import taipy as tp
tp.get_scenarios(cycle=CycleObject)

So there's actually no other motivation behind this issue?

[FlorianJacta]

I know that they already exist. Not back in my days 😄

Well, you tell me. Or @jrobinAV tells us.

[jrobinAV]

To be honest, it is hard for me to see the motivation behind all of it. Would it be faster than if I did it myself?

Without such an API, the implementation should be easy and straightforward, I agree. However, the easy solution will be to read all the scenarios and filter some of them out. In terms of performance, we could benefit from having a cycle description that can be used in the scenario query so we can only read the needed scenarios.

And I also needed to get all scenarios from a cycle object.

True.