search

dancarmoz / israel_moh_covid_dashboard_data

Data pulled from the Israeli Minsitry of Health's COVID-19 dashboard's API.
17 stars 12 forks source link

Vaccinations by Age Declining Values #12

Open RuijingZ opened 3 years ago

RuijingZ commented 3 years ago

I found that there are dramatic declining value for Vaccinations 70-90 years old and over 90+ years old on Jun 27 for both 1st and 2nd doses, which does not make sure, since these data should be cumulative. The population does not have any change as well. May I ask if you could check the validity of these declining values?

dancarmoz commented 3 years ago

Hi Ruijing, Thanks for noticing and asking. I don't really have any way to check the validity of the data, I don't have any contacts in the Ministry, I just fetch the data from the API and save it for posterity. I'm confident that the data saved in the table is the data reported by the API, unfortunately, I think this data (i.e. the cumulative distributions of data-by-age-groups) has been problematic for some time. I recently noticed a similar issue with the other tables of distributions by age and gender of infected, serious condition, intubated and deceased; these all seem to have been somewhat wonky starting from mid-April -- coinciding with when the API started returning information not only of the total cumulative numbers since the beginning of COVID-19, but also ranging back only 1, 3, 6 or 12 months. It is probably unsurprising that this change created weirdness in the cumulative data as well, though I don't know how to explain it.

This might not be related to the bug you found, though, as that field is somewhat different. I would wager that what happened with vaccination rate is simply that the past information was inaccurate, and the new and lower information is unfortunately the correct one; vaccination rates above 99% seem unlikely, to me. I'll mention that some months ago, there were some cities that were seemingly going to surpass 100% vaccination rate (including the young, non-eligible population!), and when indeed the rates were very close to 100%, the mistake was caught and the data was corrected, with the numbers of vaccinated citizens in those cities cut down dramatically. I believe it's very likely the same happened here.

If you're interested and have time to investigate the issue with the other by-age-and-gender tables, I recommend checking out previous version of the main .json file (in the python file you can find the function "fetch_histortic_data" that does this), and investigating their fields "[infected,severe,breathe,dead]ByPeriodAndAgeAndGender". I'll be very interested to know if you have any insights regarding what happened there...

dancarmoz commented 3 years ago

Yes, it seems my guess was correct -- the Ministry's SitRep from July 2nd (sent in the telegram channel, and only there to the best of my knowledge) mentions that on June 28th there was a data cleansing regarding numbers of vaccinations which moved some vaccinated between age groups, and probably is responsible for these changes as well (in fact it was on June 27th as you said, between morning and evening, but small difference).

RuijingZ commented 3 years ago

Yes, it seems my guess was correct -- the Ministry's SitRep from July 2nd (sent in the telegram channel, and only there to the best of my knowledge) mentions that on June 28th there was a data cleansing regarding numbers of vaccinations which moved some vaccinated between age groups, and probably is responsible for these changes as well (in fact it was on June 27th as you said, between morning and evening, but small difference).

Yeah, now the data revised. Thank you for investigating it!