search

lisphilar / covid19-sir

CovsirPhy: Python library for COVID-19 analysis with phase-dependent SIR-derived ODE models.
https://lisphilar.github.io/covid19-sir/
Apache License 2.0
109 stars 44 forks source link

Fixing Bug: ParserError with Population class #42

Closed SM-ins closed 4 years ago

SM-ins commented 4 years ago

Hii,

pop_data = cs.Population( "../input/world-population/API_EN.POP.DNST_DS2_en_csv_v2.csv" ) pop_data.cleaned().tail()

i am getting the error given below while running the above code...,please help if possible...

ParserError: Error tokenizing data. C error: Expected 3 fields in line 5, saw 62

With regards, Rakesh

lisphilar commented 4 years ago

Dear Rakesh(@SM-ins), Thank you for your feed-back!

cs.Population does not input the CSV files directly downloaded from THE WORLD BANK. We need DataLoader class to use them. Please run the following codes with the latest version (>2.3.0).

data_loader = cs.DataLoader("../input")
pop_data = data_loader.population()
pop_data.cleaned().tail()

Type of pop_data is equal to cs.Population and "locations_population.csv" will be saved in "../input" directory. This is different from "API_EN.POP.DNST_DS2_en_csv_v2.csv", but well organized.

Best Regards, Lisphilar

lisphilar commented 4 years ago

I will change the error statement. Before: ParserError: Error tokenizing data. C error: Expected 3 fields in line 5, saw 62 After: KeyError: Raw data of PopulationData must have Country.Region, Province.State, Population but not included.

SM-ins commented 4 years ago

Dear Rakesh(@SM-ins), Thank you for your feed-back!

cs.Population does not input the CSV files directly downloaded from THE WORLD BANK. We need DataLoader class to use them. Please run the following codes with the latest version (>2.3.0).

data_loader = cs.DataLoader("../input")
pop_data = data_loader.population()
pop_data.cleaned().tail()

Type of pop_data is equal to cs.Population and "locations_population.csv" will be saved in "../input" directory. This is different from "API_EN.POP.DNST_DS2_en_csv_v2.csv", but well organized.

Best Regards, Lisphilar

Dear Rakesh(@SM-ins), Thank you for your feed-back!

cs.Population does not input the CSV files directly downloaded from THE WORLD BANK. We need DataLoader class to use them. Please run the following codes with the latest version (>2.3.0).

data_loader = cs.DataLoader("../input")
pop_data = data_loader.population()
pop_data.cleaned().tail()

Type of pop_data is equal to cs.Population and "locations_population.csv" will be saved in "../input" directory. This is different from "API_EN.POP.DNST_DS2_en_csv_v2.csv", but well organized.

Best Regards, Lisphilar

Dear Lisphilar, Thankyou so much for the response.

i tried to learn Your following code:-

data_loader = cs.DataLoader("../input")

pop_data = data_loader.population() pop_data.cleaned().tail()

but now i am getting a new error:- OSError: [Errno 30] Read-only file system: '/kaggle/input/locations_population.csv'

With regards, Rakesh

lisphilar commented 4 years ago

Dear Rakesh(@SM-ins ), WIth the error code, I suppose that you are using this package in Kaggle Notebook. When you use this package in Kaggle, please add the following datasets manually with "Add Data" button (I think you have done).

Then, please use the following data.

import covsirphy as cs
# The number of cases (JHU)
jhu_data = cs.JHUData("/kaggle/input/novel-corona-virus-2019-dataset/covid_19_data.csv")
# (Optional) The number of cases in Japan
japan_data = cs.CountryData("/kaggle/input/covid19-dataset-in-japan/covid_jpn_total.csv", country="Japan")
japan_data.set_variables(
    date="Date", confirmed="Positive", fatal="Fatal", recovered="Discharged", province=None
)
# Population in each country
population_data = cs.PopulationData(
    "/kaggle/input/covid19-global-forecasting-locations-population/locations_population.csv"
)

The codes I suggested in the previous comment is for standard users/developers.

I apologize for confusing you. Thank you.

SM-ins commented 4 years ago

Dear Rakesh(@SM-ins ), WIth the error code, I suppose that you are using this package in Kaggle Notebook. When you use this package in Kaggle, please add the following datasets manually with "Add Data" button (I think you have done).

Then, please use the following data.

import covsirphy as cs
# The number of cases (JHU)
jhu_data = cs.JHUData("/kaggle/input/novel-corona-virus-2019-dataset/covid_19_data.csv")
# (Optional) The number of cases in Japan
japan_data = cs.CountryData("/kaggle/input/covid19-dataset-in-japan/covid_jpn_total.csv", country="Japan")
japan_data.set_variables(
    date="Date", confirmed="Positive", fatal="Fatal", recovered="Discharged", province=None
)
# Population in each country
population_data = cs.PopulationData(
    "/kaggle/input/covid19-global-forecasting-locations-population/locations_population.csv"
)

The codes I suggested in the previous comment is for standard users/developers.

I apologize for confusing you. Thank you.

Dear Lisphilar(@lisphilar ) Thankyou so much for taking out Your valuable time and solving my problem.You are right ..i am using kaggle notebook..,i am sorry..i should have mentioned it earlier...

With regards, Rakesh

lisphilar commented 4 years ago

Dear Rakesh (@SM-ins ), Thank you very much for your confirmation and using this package.

Finally, I fixed the bug with #50 and #52 We can now use the following simple codes in Kaggle.

data_loader = cs.DataLoader("kaggle/input")
jhu_data = data_loader.jhu(
    local_file="/kaggle/input/novel-corona-virus-2019-dataset/covid_19_data.csv"
)
japan_data = data_loader.japan(
    local_file="/kaggle/input/covid19-dataset-in-japan/covid_jpn_total.csv"
)
population_data = data_loader.population(
    local_file="/kaggle/input/covid19-global-forecasting-locations-population/locations_population.csv"
)

Thank you for your cooperation.

lisphilar commented 4 years ago

Dear Rakesh @SM-ins , I'm sorry, but the code I showed at the last comment was wrong. I edited the comment and the correct version is here.

data_loader = cs.DataLoader("kaggle/input")
jhu_data = data_loader.jhu(
    local_file="/kaggle/input/novel-corona-virus-2019-dataset/covid_19_data.csv"
)
japan_data = data_loader.japan(
    local_file="/kaggle/input/covid19-dataset-in-japan/covid_jpn_total.csv"
)
population_data = data_loader.population(
    local_file="/kaggle/input/covid19-global-forecasting-locations-population/locations_population.csv"
)

We can use data loader class, but we need to specify local_file argument.

SM-ins commented 4 years ago

Dear Lisphilar,

Thankyou so much for kind response..

Best regards, Rakesh

On Thu, Jul 2, 2020 at 6:50 PM Lisphilar notifications@github.com wrote:

Dear Rakesh @SM-ins https://github.com/SM-ins , I'm sorry, but the code I showed at the last comment was wrong. I edited the comment and the correct version is here.

data_loader = cs.DataLoader("kaggle/input")jhu_data = data_loader.jhu( local_file="/kaggle/input/novel-corona-virus-2019-dataset/covid_19_data.csv" )japan_data = data_loader.japan( local_file="/kaggle/input/covid19-dataset-in-japan/covid_jpn_total.csv" )population_data = data_loader.population( local_file="/kaggle/input/covid19-global-forecasting-locations-population/locations_population.csv" )

We can use data loader class, but we need to specify local_file argument.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lisphilar/covid19-sir/issues/42#issuecomment-653001722, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP6Y4PEALNPGTYD522W45R3RZSCTRANCNFSM4OLG5AGQ .

lisphilar commented 4 years ago

Thank you and I will close this issue because the bugs were fixed.

SM-ins commented 4 years ago

Dear @lisphilar/covid19-sir reply@reply.github.com ,

Thanks..,,actually whenever i was taking 6 phases in case of India then the first phase was getting stuck every time..,so reduced it to 5 phase and minimise the gaps between the start date and end date of consecutive phases..,then it ran but still I think that it's not the proper way...what do you think...??

With regards, Rakesh

On Wed, Jul 22, 2020 at 7:10 PM Lisphilar notifications@github.com wrote:

Dear Rakesh, Great! How did you find the way to adjust phase information? This will be helpful to automate the process with this package.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/lisphilar/covid19-sir/issues/128#issuecomment-662459549, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP6Y4PGDQ5QOXJGKOMQCT53R43T5LANCNFSM4PDPXH4A .