investigate issue of rbp forecasted to below 0 in extreme scenarios

[kkarrancsu]

Code snippet to produce RBP < 0

import numpy as np
import datetime
import scenario_generator.curated as curated
from mechafil.data import get_historical_network_stats, get_sector_expiration_stats
from mechafil.power import forecast_power_stats, build_full_power_stats_df

SIM_START_LAG_DAYS = 10
current_date = datetime.datetime.now().date() - datetime.timedelta(days=SIM_START_LAG_DAYS)
current_day = (current_date - datetime.date(2020, 10, 15)).days

start_date = datetime.date(2021, 3, 15)
start_day = (start_date - datetime.date(2020, 10, 15)).days

forecast_lenght = 365*2
end_day = current_day + forecast_lenght
end_date = current_date + datetime.timedelta(days=forecast_lenght)

duration = 360

scenario_params = curated.forecast_pessimistic_scenario(forecast_lenght)
rb_onboard_power_vec = scenario_params['rb_onboard_power']
renewal_rate_vec = scenario_params['renewal_rate']
filplus_rate_vec = scenario_params['filplus_rate']

res = get_sector_expiration_stats(start_date, current_date,end_date)
rb_known_scheduled_expire_vec = res[0]
qa_known_scheduled_expire_vec = res[1]
known_scheduled_pledge_release_full_vec = res[2]

fil_stats_df = get_historical_network_stats(start_date,current_date,end_date)
current_day_stats = fil_stats_df[fil_stats_df["date"] >= current_date].iloc[0]

rb_power_zero = current_day_stats["total_raw_power_eib"] * 1024.0
qa_power_zero = current_day_stats["total_qa_power_eib"] * 1024.0

rb_power_df, qa_power_df = forecast_power_stats(
        rb_power_zero,
        qa_power_zero,
        rb_onboard_power_vec,
        rb_known_scheduled_expire_vec,
        qa_known_scheduled_expire_vec,
        renewal_rate_vec,
        filplus_rate_vec,
        duration,
        forecast_lenght
)
rb_power_df["total_raw_power_eib"] = rb_power_df["total_power"]/1024.0
qa_power_df["total_qa_power_eib"] = qa_power_df["total_power"]/1024.0

power_df = build_full_power_stats_df(
        fil_stats_df,
        rb_power_df,
        qa_power_df,
        start_date,
        current_date,
        end_date,
)
print(np.min(power_df.total_raw_power_eib))
...

[kkarrancsu]

I believe the root this issue is that our simulation uses a constant sector-duration parameter. In reality, sectors may have longer durations. The image below shows the various powers. All of the data is historical (i.e. no projections). The dotted line is there to indicate the start of simulation, but the scheduled expirations are also historical - recall that we know scheduled-expirations upto a certain future date based on sector durations. However, we only know the onboarded and renewed power upto the present date.

Examine the purple dotted line, which has a length equal to the sector duration. At the end of the purple line, we see that the scheduled expired power > the total onboarded+renewed power. Secondly, I think that the power scheduled to expire after (forecast_start + duration) is power from sectors that have duration > simulation's configured duration. But, this is inconsistent w/ the simulation and causes negative power in extremely pessimistic scenarios of renewal rates and onboarding power. One solution is to not use the scheduled power expire vector after time forecast_start+duration. Before & after plots of power with this implementation shown below.

Interested to hear your thoughts @misilva73 @tmellan

Original

With proposed solution

[misilva73]

At the end of the purple line, we see that the scheduled expired power > the total onboarded+renewed power.

I think this happens because the known sectors don't all have the same duration. In order words, the vector rb_known_scheduled_expire_vec includes power from sectors renewed or onboarded in a different date than 01-2023.

[misilva73]

I think that the power scheduled to expire after (forecast_start + duration) is power from sectors that have duration > simulation's configured duration. But, this is inconsistent w/ the simulation and causes negative power in extremely pessimistic scenarios of renewal rates and onboarding power

I am not following this. By forecast_start do you mean the start of power forecasting? I.e. the current date?

And why are known sectors with duration > simulation's configured duration a problem/inconsistency?

Also, the sectors being "onboarded" after the current date (i.e. the "modelled sectors"), will always be scheduled to expire after forecast_start + duration and their duration is by definition the simulation's configured duration.

[misilva73]

One solution is to not use the scheduled power expire vector after time forecast_start+duration.

If we don't use the scheduled power expire vector, what do we use instead? I.e., what is the solution?

[misilva73]

I am curious to know what parameters are used in the code snippet to produce RBP < 0:

scenario_params = curated.forecast_pessimistic_scenario(forecast_lenght)
rb_onboard_power_vec = scenario_params['rb_onboard_power']
renewal_rate_vec = scenario_params['renewal_rate']
filplus_rate_vec = scenario_params['filplus_rate']

Do you know the values used?

[kkarrancsu]

forecast_start

Date at which forecasting starts.

If we don't use the scheduled power expire vector, what do we use instead? I.e., what is the solution?

This was poorly written. I should have said, my proposed solution is to not use rb_known_scheduled_expire_vec after forecast_start + duration. This means that only modeled sectors contribute to SE power after t=forecast_start+duration I think this resolves the other comments you were mentioning, but let me know if not?

Input values