The PfLine and PfState classes have the .loc[] property, which is inherited from pandas. It allows us to select from/until a certain row. E.g., if df is a pandas DataFrame, we can do df.loc['2020':'2022'].
This is inclusive-inclusive, meaning, both the left and the right limit are included in the selection. Let's say df has hourly values:
df.loc['2020-01-01':'2022-01-01'] -> The first row has timestamp '2020-01-01 00:00' and the final row has '2022-01-01 23:00'.
df.loc['2020':'2022'] -> The first row has timestamp '2020-01-01 00:00' and the final row has '2022-12-31 23:00'.
Problem
This goes contrary to common slicing operations in python, which do not include the endpoint.
It is unpractical too. Let's say we have a price-pfline pfl1 spanning the whole 2020, 2021, and 2022, and pfl2 spanning the whole 2022, 2023 and 2024. And we want to combine them into 1 pfline spanning 2020, 2021, 2022, 2023 and 2024. Here we focus on slicing the portfolio lines so that they have no overlap but also no gap between them. (How we combine them into a new pfline is a different issue.) Let's say we want the split to be so, that everything before 2022 must be from pfl1, and everything during/after 2022 from pfl2. Using .loc[], we must use:
pfls_to_concat = [pfl1.loc[:'2021'], pfl2.loc['2022':]]. Notice how we cannot use the same timestamp to make the selection in both.
Or we can do pfls_to_concat = [pfl1.loc[:'2021-12-31 23:00'], pfl2.loc['2022-01-01 00:00':]]. This is even worse, because we have to know the frequency of the pflines and stay one unit away from the limit. We cannot use pfl1.loc[:'2022-01-01 00:00'], because this timestamp would be included twice in the resulting list.
Solution
Add a property (maybe called .slice[]?), which uses the more common slicing convention, where .slice[a:b] includes a but excludes b. The same issue as above then becomes:
Or we can do pfls_to_concat = [pfl1.slice[:'2022-01-01 00:00'], pfl2.slice['2022-01-01 00:00':]].
Much better.
Notes
The method should interpret both a and b as timestamps, and not as time periods. So, specifying '2022', this is not the entirety of 2022, but actually the start of 2022. It should turn it into '2022-01-01 00:00'.
We can probably just use a mask under the hood, i.e., mask = a <= self.index < b and then return a PfLine(self.df[mask]).
In pandas, pd.DataFrame[] accepts many arguments. E.g., a column name selects that series. But it also accepts a slice - df['2021':'2023'] is the same as df.loc['2021':'2023']. We don't want to do this, because we will have 2 slicing methods: .loc[] (inclusive-inclusive) and .slice[] (inclusive-exclusive). But it would be good for the user if we can detect it if a slice is passed into pfl[...], and give a useful Exception ("to make a slice, use .slice[] (recommended) or .loc[]. slice[] uses the common python convention and excludes the right timestamp; loc[] uses the pandas-convention and includes the right timestamp.")
Don't forget tests and to add a section to the documentation ;)
Pizza2Pizza
commented
6 months ago
Context
The
PfLineandPfStateclasses have the.loc[]property, which is inherited from pandas. It allows us to select from/until a certain row. E.g., ifdfis a pandas DataFrame, we can dodf.loc['2020':'2022'].This is inclusive-inclusive, meaning, both the left and the right limit are included in the selection. Let's say
dfhas hourly values:df.loc['2020-01-01':'2022-01-01']-> The first row has timestamp '2020-01-01 00:00' and the final row has '2022-01-01 23:00'.df.loc['2020':'2022']-> The first row has timestamp '2020-01-01 00:00' and the final row has '2022-12-31 23:00'.Problem
This goes contrary to common slicing operations in python, which do not include the endpoint.
It is unpractical too. Let's say we have a price-pfline
pfl1spanning the whole 2020, 2021, and 2022, andpfl2spanning the whole 2022, 2023 and 2024. And we want to combine them into 1 pfline spanning 2020, 2021, 2022, 2023 and 2024. Here we focus on slicing the portfolio lines so that they have no overlap but also no gap between them. (How we combine them into a new pfline is a different issue.) Let's say we want the split to be so, that everything before 2022 must be frompfl1, and everything during/after 2022 frompfl2. Using.loc[], we must use:pfls_to_concat = [pfl1.loc[:'2021'], pfl2.loc['2022':]]. Notice how we cannot use the same timestamp to make the selection in both.pfls_to_concat = [pfl1.loc[:'2021-12-31 23:00'], pfl2.loc['2022-01-01 00:00':]]. This is even worse, because we have to know the frequency of the pflines and stay one unit away from the limit. We cannot usepfl1.loc[:'2022-01-01 00:00'], because this timestamp would be included twice in the resulting list.Solution
Add a property (maybe called
.slice[]?), which uses the more common slicing convention, where.slice[a:b]includes a but excludes b. The same issue as above then becomes:pfls_to_concat = [pfl1.slice[:'2022'], pfl2.slice['2022':]].pfls_to_concat = [pfl1.slice[:'2022-01-01 00:00'], pfl2.slice['2022-01-01 00:00':]].Much better.
Notes
aandbas timestamps, and not as time periods. So, specifying '2022', this is not the entirety of 2022, but actually the start of 2022. It should turn it into '2022-01-01 00:00'.mask = a <= self.index < band then return aPfLine(self.df[mask]).pd.DataFrame[]accepts many arguments. E.g., a column name selects that series. But it also accepts a slice -df['2021':'2023']is the same asdf.loc['2021':'2023']. We don't want to do this, because we will have 2 slicing methods:.loc[](inclusive-inclusive) and.slice[](inclusive-exclusive). But it would be good for the user if we can detect it if a slice is passed intopfl[...], and give a useful Exception ("to make a slice, use.slice[](recommended) or.loc[].slice[]uses the common python convention and excludes the right timestamp;loc[]uses the pandas-convention and includes the right timestamp.")