Comparing Scenario analysis estimate & simulation for the same dates

[jodobear]

What we need to document?

I want to compare scenario analysis for the same dates e.g.:

1. Get scenario analysis for 1 Mar 2020 to 30 June 2020
2. Simulate scenario from 1 July 2020 to 31 August 2020
3. Get scenario analysis for 1 July 2020 to 31 August 2020
4. Compare results

Basically I want to be able to compare simulation vs actual data for the same dates and see how the model differs from the actual data. Is that possible?

I tried doing it from the same dates but once the model is created for a date i cannot use it on the same dates again. Probably i'm missing something

Any help will be greatly appreciated. Thank you for the amazing work.

[lisphilar]

Dear @jodobear , Thank you for sharing your great idea!

It is necessary to compare the two following scenarios. Is this your point?

1. Perform parameter estimation with records to 31Aug2020
2. Perform parameter estimation with records to 30Jun2020 and simuate the number of cases from 01Jul2020 to 31Aug2020 with the assumption that the parameter values are the same as that of the final phase before 01Jul2020

If so, we can use the next codes in the latest version 2.8.1

import covsirphy as cs
data_loader = cs.DataLoader("input")
jhu_data = data_loader.jhu()
population_data = data_loader.population()
# As an example, Japan dataset will be used here
snl = cs.Scenario(jhu_data, population_data, "Japan")
# Set the range of records
snl.first_date = "01Mar2020"
snl.last_date = "31Aug2020"
# Set main scenario (Estimation with records to 31Aug2020)
snl.trend(include_init_phase=True)
snl.separate("01Jul2020")
snl.estimate(cs.SIRF)
# Set sub scenario (Estimation with records to 30Jun2020 and simulation to 31Aug2020)
# Copy main, delete phases after 31Jun2020, and add new phase from 01Jul2020 to 31Aug2020
snl.clear(name="sub")
snl.delete(["2nd", "3rd", "4th", "5th", "6th", "7th"], name="sub")
snl.add(name="sub")
# Compare scenarios
snl.summary()
snl.describe()
snl.history("Rt")
snl.history("Infected")

I know these codes may be confusing. If you have time, please help me with .ipynb documentation (or creating new method for these codes).

Thank you!

[jodobear]

Dear Lisphilar,

Thank you for you prompt reply and with code.

Basic idea is to validate various models against real world data. Basically, I want to get parameters until a point (my first comment is an example), take those parameters and estimate the next few months using those parameters. Once we get all the numbers(R0, S, I, R, D) we compare what we find in the real data.

This will help us understand the model better.

I will try the code and report back. And would be more than happy to contribute on this once I understand what this does.

Cheers!

[jodobear]

When I run snl.estimate(cs.SIR) I get the error:

ValueError: @end_date must be the same as/over 01Sep2020, but 31Aug2020 was applied.

So, I changed it to 01Sep2020 then I got:

ValueError: @end_date must be the same as/over 02Sep2020, but 01Sep2020 was applied.

This is the error I got stuck at before as well. Not sure why the error shows up. Any ideas?

[lisphilar]

Dear @jodobear , Please clarify the code line with error.. ValueError with this statement is sometimes raised when name argument was not specified or the phase for the date has been registered.

[jodobear]

it gets triggered when I run snl.add(name="sub")

[lisphilar]

Please run 'snl.summary()' If a phase was registered for 31Aug2020, please skip the line you mentioned. Sorry, I could not reproduced the error in my local snvironment and Colab.

P.S. Good night from Japan.

[jodobear]

I reset the runtime on colab and ran the whole code again, but get the same error and the last phase in snl.summary() is 27Jul2020 to 31Aug2020

[lisphilar]

Dear @jodobear , Here is my notebook in GitHub/Gist. https://gist.github.com/lisphilar/7d10d7f257cc6c7df82ed3146fcd8404

Please open the notebook with Google Colab and compare the outputs with your results.

[jodobear]

Dear @lisphilar ,

I checked the notebook and it is working, not sure why when i did, it didn't work.

So, here we have 2 scenarios > Main & sub, where:

1. Main is the full estimation of the data and parameters are calculated 31Aug2020
2. We create a separate pin at 01Jul2020 label it scenario sub and run an estimate from 01Jul2020 to 31Aug2020 using the same parameters calculated in 1(Main)

Am I understanding it correctly? Also, I'm still not clear on what add & delete methods do. Is this where we remove the data after July and add later to simulate using the parameters calculated before?

Again thank you for your help. Once I understand this, will update the docs. Cheers!

[lisphilar]

Dear @jodobear , Yes, main/sub scenarios and delete/add methods act as you mentioned. The details of delete/add methods are as follows.

<Scenario.delete(phases=None, name='Main')>

When 1st (01Apr - 30Jun), 2nd (01Jul - 31Jul) and 3rd phase (01Aug - 31Aug) are registered, Scenario.delete(phases=["2nd"]).summary() returns 1st phase (01Apr - 31Jul) and 2nd phase (01Aug - 31Aug). 01Jul - 31Jul will be included in 1st phase.

When 1st (01Apr - 30Jun), 2nd (01Jul - 31Jul) and 3rd phase (01Aug - 31Aug) are registered, Scenario.delete(phases=["3rd"]).summary() returns 1st phase (01Apr - 30Jun) and 2nd phase (01Jul - 31Jul). 01Aug - 31Aug will NOT be included.

<Scenario.add(name='Main', end_date=None, days=None, population=None, model=None, **kwargs)>

When the last date of the registered phases (1st phase) is 30Jun, Scenario.add(end_date="31Jul2020") appends 2nd phase with 01Jul - 31Jul.

When the last date of the registered phases (1st phase) is 30Jun, Scenario.add(days=10) appends 2nd phase with "from 01Jul to 10Jul".

When the last date of the registered phases (1st phase) is 30Jun and the last date of records (Scenario.last_date) is "31Aug2020", Scenario.add(end_date=None, days=None) appends 2nd phase with 01Jul - 31Aug.

When the parameter values were set for the last registered phase, the parameter values of the new phase will be the same as those of the last registered phase.

When population is not None, the population value of the new phase will be the specified value.

Scenario.add(end_date="31Jul", sigma=0.2) appends a new phase with sigma=0.2

When model is applied, the parameter values will be removed and ODE model will be changed.

Thank you!

[lisphilar]

Updated API reference of Scenario.delete()

[lisphilar]

Dear @jodobear , This is just a notificaion regarding new version 2.8.2

Initial phase (0th phase) will be enabled by Scenario.trend() as default (disabled in version <= 2.8.1). I think this does not require code change in your analysis, but "0th phase" will be included in the out put of Scenario.trend().summary()

Thank you.

[lisphilar]

Dear @jodobear , Please let me know if you need more information regarding this issue. I need your help on this project.

[jodobear]

Dear @lisphilar ,

Thank you for the update. I have not been able to work on this the last couple of weeks. I'm very happy to help, what do we need now?

[lisphilar]

Dear @jodobear , Thank you for coming again. The document discussed above (retrospective analysis) is still under discussion. Did you catch the image of Scenario.add() and Scenario.delete()?

[lisphilar]

Scenario.retrospective()beginning_date, model, control="Main", target="Target", **args) will be included in version 2.9.0

snl = cs.Scenario(jhu_data, population_data, "Japan")
snl.retrospective("01Sep2020", model=cs.SIRF, control="Main", target="Retrospective")
# Show summary
print(snl.summary())
# SHow history
snl.history("Infected")

[lisphilar]

This will be documented in the last section of https://lisphilar.github.io/covid19-sir/usage_quick.html