Closed CSSEGISandData closed 4 years ago
@CSSEGISandData also regarding invalid numbers for France, it appears that the daily reports for 03-23-2020.csv
for French Polynesia
includes also the values for the France
mainland. (The series
files seems to be correct.)
For example:
03-22-2020.csv
:
,,,France,3/22/20 23:45,46.2276,2.2137,16018,674,2200,13144,France
,,French Polynesia,France,3/22/20 23:45,-17.6797,149.4068,15,0,0,15,"French Polynesia, France"
03-23-2020.csv
:
# France missing
,,French Polynesia,France,2020-03-23 23:19:21,-17.6797,-149.4068,19874,860,2200,16814,"French Polynesia, France"
03-24-2020.csv
:
,,,France,2020-03-24 23:37:15,46.2276,2.2137,22304,1100,3281,17923,France
,,French Polynesia,France,2020-03-24 23:37:15,-17.6797,-149.4068,25,0,0,25,"French Polynesia, France"
I.e. for 23rd the France
row is missing, and the numbers for French Polynesia
jump drastically from 22nd, but then go back again for 24th.
This is the data source you should be using: https://www.gouvernement.fr/info-coronavirus/carte-et-donnees
also (if this is better than opening a new case... :), the 04-09-2020.csv daily file has an odd French entry:
00250,,,France,2020-04-09 23:02:19,46.2276,2.2137,117749,12210,23206,82333,",,France"
(that leading "00250" seems bogus.)
you're back to the wrong number:
Just to have the correct (i.e. official from other sources) numbers here:
The data here says: 117749 Number in other data sources [1,2]: 86334
[1] https://dashboard.covid19.data.gouv.fr/ [2] https://github.com/opencovid19-fr/data/blob/master/ministere-sante/2020-04-09.yaml
If France were releasing confirmed data with the same accuracy they do for hospital cases, things would be so much simplier, but hey, that's how we do in France with official data, never simple... Thanks for your great work @CSSEGISandData, it must not be easy dealing with the different way of reporting data in every different country!
@boogheta Perhaps as french yourself, could you explain the situation of nursing homes, instead of inappropriate comment.
@JiPiBi I'm not sure how this is inappropriate. France has a long history of cultural lack of transparency and releasing hardly valid data (see for instance companies register data...). The lack of transparency from the government on nurses home situation makes it hard to explain properly, but basically people are not all tested, and as soon as a spread is observed in a nursing home, all sick people in this home are accounted. This does not legitimate government not publishing detailed figures coming from these since they do account them in their total national confirmed cases and in a separate national deaths count.
The lack of transparency from the government on nurses home situation makes it hard to explain properly, but basically people are not all tested, and as soon as a spread is observed in a nursing home, all sick people in this home are accounted. This does not legitimate government not publishing detailed figures coming from these since they do account them in their total national confirmed cases and in a separate national deaths count.
Do you mean that "cas confirmés" is over-reported here: https://www.gouvernement.fr/info-coronavirus/carte-et-donnees ?
I wouldn't go as far as "over-reported", but according to the way it is accounted in nurses home and what the government announced, it should rather be called "cas confirmés ou fortement suspectés" yes.
@boogheta I disagree, the cases in nursing homes are separatedly presented on the official dashboard and not aggregate in the national confirmed cases for the reasons you gave. Secondly the people from ESMS who died in hospitals are in the hospital death counter and the people died outside hospital are in the death global counter.
@JiPiBi No, according to official speeches, the cases of deceased are presented separately only, not the cases of confirmed. This is coherent with the fact that hospitals do not report data on confirmed: deceased must be added from both sources (hospitals and ehpad), whereas there is no breakdown for confirmed (neither by source nor by region), only global figures accounted in different ways and which might overlap... This is exactly the point @CSSEGISandData is explaining in this issue and why they are reverting (tonight?) to the good value which is around 90 000 and not 110 000 as was reported over the past week but not touching the number of deceased which is the good one. You can read more about it in French here: https://www.liberation.fr/checknews/2020/04/05/covid-19-pourquoi-des-sites-evoquent-90-000-cas-en-france-contre-68-000-au-bilan-officiel_1784232
@boogheta No about what ?? What did I say that is not coherent with the following yesterday's dashboard
As far as i understand it, the ESMS/EHPAD cases are not in fact confirmed cases, but consist in fact in part of speculative cases. therefore adding them to a data field named Confirmed is deeply problematic. If there is no valid conclusive official count of definitely confirmed cases outside the official number, there is nothing that can be added without setting fire to any meaning.
Those 31415 unconfirmed cases do by definition include an undisclosed amount of false positive cases. Including these in the total of confirmed cases does noet adhere to any standard, scientific, journalistic or otherwise.
The ever-changing ad-hoc-reasoning by JHU CSSE is deeply worrying.
@yetzt You are right , the 31415 are not for the moment sorted between confirmed and suspected , a ratio of 1/3 of confimed has been annouced yesterday and we are waiting on more accurate values In France there are about 7000 nursing homes , which where not in the informatic network of health system . This is the explanation of huge increase in death cases because many of these old people died very quickly in the nursing homes and not in the hospitals and the true values were not known by health system, only by city services . It seems that the people could be in quite good form and some hours later became very sick and die People after a certain age can't support ICU even if there are in an hospital. Imagine to be 2 or 3 weeks unconscious on the belly . Even for young people it is hard to recover, it may last weeks or moths . In hospital about 25-30% of people under 80's are in ICU . Above 80's only about 10% are in ICU . And they represent the main part of deaths ...
Update
I have another question: What is the total number of recovered? "Retours à domicile" means patients released from hospital. What about those who had the virus but were never hospitalised? Do we just use this? recovered = total infected - total dead - active cases
Can we find a figure for total active cases somewhere?
Recovered here is clearly recovered after being hospitalyzed.
People who had mild symptoms were asked to stay at home : mainly not tested, in tight contact with their doctor to see if the sickness is evoluting.
In the family , there is also a possibility that another person is positive but these values are not known and if they recover at home these values are also not known and probably will never been known.
So as we dont know how many people where infected , the notion of recovered means nothing for the moment . Only serologic tests could give a real idea of the situation .
See on worldometers the informations about this experiment in Germany :
A study by the University of Bonn has tested a randomized sample of 1,000 residents of the town of Gangelt (an epicenter of the outbreak in Germany) and found that 2% of the population was currently infected and 14% were carrying antibodies suggesting that they had already been infected — whether or not they experienced any symptoms. Eliminating an overlap between the two groups, the team concluded that 15% of the town have been infected with the virus [source]
If these findings are correct, Germany’s actual death rate could be as low as 0.22% (2,607 deaths / (2,607 cases that have resulted in death + 1,172,000 cases that have resulted in recovery)). Assuming 14% of the German population of 83,700,000 (1,172,000 people) have been infected and have recovered.
Source : https://spectator.us/covid-antibody-test-german-town-shows-15-percent-infection-rate/
The notion of recovered is very clear. There are many countries in Europe where people who tested positive and had mild symptoms were made to self-isolate at home, then tested again periodically, and finally declared recovered when they tested negative (the Prince of Wales is one example). They were then added to the tally of recovered cases.
If France only considers as recovered those patients who were released from hospital, it is under-reporting recoveries.
You are right. But Prince of Wales is not the good example. John Doe on SmallCity has not the same attention :-)
People that recovered alone at home have been rarely tested positive, because lack of tests at the beginning. I m not justifying, I am explaining the values.
Just to make it 100% clear. Here in France they are currently mainly testing people admitted to hospital with advanced symptoms, front-line medical staff (plus politicians etc.). The general public are not being tested even if symptomatic. If a nursing home resident falls ill, is hospitalized and tests positive then the whole home is classed as 'possible'.
The "recovered" cannot include people who self-isolated and got well since they were never confirmed in the first place (French logic).
The govt. have announced the start of 'massive' testing in nursing homes, so those 'possibles' will gradually turn into 'confirmed' - this means that we should not include them in the statistics at the moment as they will eventually end up being counted twice!
The French count includes ALL confirmed cases as per this graph (from https://www.gouvernement.fr/info-coronavirus/carte-et-donnees at 01:00 CEST 2020-04-11)
With this key
Translated it reads (from top to bottom)
I hope that helps clarify the matter
Thank you.
The different reporting methods make forecasting harder. I would have expected standardised reporting at least across the EU, or what remains of it.
@Clumpton Nothing to add to your explanations.
Remarks:
I hesitated a bit to understand autre hospitalisation , suggestion to call hosp en rea, hosp hors rea, and final autre: non hosp or confirmes a domicile. But details
Add décès in hosp+ recovered+ currently hosp and compare to 65420. I dont understand the result.
@ivandebono Agree with the reporting differences making it hard to compare. For example, knowing new ICU and new hosp is very important now in France and Europe. At the beginning cases and deaths are important, in the middle, the exponential increase and quantity of beds and ICU at the peek the new hosp and new ICU, and after I suppose all the dead includind surmortality by class of age and the percentage of infected people
I think WHO should now propose common reporting.
@JiPiBi
* Add décès in hosp+ recovered+ currently hosp and compare to 65420. I dont understand the result.
I don't understand where you get 65240 from?
Look above, I put yesterday the most recent informations. There are 6 colored cells from SPF. My purpose is to understand the value that counts all people that went to hospital in the upper middle cell.
@JiPiBi OK I see now, that 65240 cell is the total over the pandemic, if you look at the 3rd line it says "dont 31267 en cours" (of which 31267 are currently in hospital). Add 32167 (current), 24932 (number discharged) and 8598 (hospital deaths) you get 64797 - a disparity of 443 - perhaps those are cases that were hospitalised and tested negative?
You got my issue to understand this gap, except that the current value for hosp is 31267(not 32167) and the difference is 623 persons. It was almost the same 2 days ago. There is an hypothesis that some of them could be linked to people sent to other regions, because it seems that they are counted again as new for the new hospital. Question asked without answer for the moment.
Oops sorry about that slip. The hypothesis of the hospital changes counting twice sounds quite likely. On second thoughts, maybe it is due to deaths sometimes being reported 1-2 days later than hospitalizations? For example there have been several hospitals that have had to resort to pen & paper records recently due to cyber-attacks.
The "recovered" cannot include people who self-isolated and got well since they were never confirmed in the first place (French logic).
Can you confirm, then, that everyone who tested positive was admitted to hospital.
And if you include "possible" cases with the grand total of infections, why not include "possible" cases with the grand total of recoveries? In statistics we call it systematic bias.
@ivandebono
Can you confirm, then, that everyone who tested positive was admitted to hospital.
Of course not, many have self-isolated, but SO FAR in France the huge majority of tests have been done on people exhibiting symptoms that prompted hospitalization. I am fully aware that the actual situation in France is forcibly far worse than indicated by the statistics due to insufficient testing. The grand total of recoveries is (as stated above, and as far as I know) those who have been discharged from hospital. I suspect that the govt. doesn't currently have the faintest idea of how many have recovered in nursing homes or in self-isolation, so there are no possible cases to include in recoveries. So the systematic bias exists only if you insist on including a figure for "possible" cases in the confirmed cases while being unable to include or estimate the "recovered possible cases" in recoveries.
Please remember that these "possible" cases are a number pulled out of the air by a minister. Probably (N° of Homes with a case) (average occupants in homes) (90% infection rate).
Anyway if Johns Hopkins wishes to insist that France has 133k while the French govt. says 95k they will only lose all credibility in France, all while increasing the paranoia that the French people have about being lied by their govt. That's me finished.
Do you really think that the number of confirmed or recovered is so important now, you dont really know both values... Even deaths are unknown, home dead people covid suspected are not tested... You will only measure later surmortality...
IMHO what is more important now is the number of hospitalyzed in or out ICU, and the evolution of these values, day after day.
@JiPiBi
I agree I don't know why people are wittering about possible recovered etc.
Of course it's important. When you try to fit epidemiological models to the data, you want your data to be as accurate as possible. That's basic statistical science.
If you're making policy based on the fitted models, it becomes even more vital.
But when data are not available, what can you do ?
Do you really think that chinese data were faithfull and helped other countries to prepare themselves? At first , people said, its China, they have strange habits with animals and the epidemic kills only old and already very sick people. Comorbidities as they said.
After, it's the turn of Italy, people say , they have no modern hospital and they have so many old people and olds live in the same house than youngs. Italian are also so latin... But Lombardia is not a third world region... And after in France you meet difficulties with a region, you discover that you have not enough masks, not enough tests, and people in nursing homes are not well protected.
It seems that countries never learn from the previous infected ones if you look at the situation in the US. They only know to apply confinment and they add economic troubles to health troubles.
In Japan , there is no confinment, there are no many tests , the cities are crowdy and they have few deaths.
Italy : many tests, many deaths Germany: many tests, few deaths Japan: few tests, few deaths.
Is there a model fitting all that situations ?
Moreover, what seems important in these epidemic is local red hot situations : every country has its Wuhan and its Hubei. In France we have 3 or 4 regions with difficulties. In Italy, there is Lombardia and Bergamo and they dont represent the whole country. So you need to be more specific than the country scale.
And now apart from statistics, there is a simple fact , how do you deal with a huge arrival of very sick people in the same hospital, you are in a scale of time where every hour is important ?
So I dont see for the moment how data can help to deal with such a crisis, if you are not prepared culturally at first and well equiped in second .
But strangely , small countries like Portugal, Croatia , Austria have very good results. Can data help to understand all that diversity and make usefull forecasts ? Some days ago, we saw that the results were better in Italy. But even Italian descent is quite chaotic. Could data explain that?
Pls , Look at the new hosp evolution in Italia.(that is new people hospitalyzed every day)
Can model show the real tendancy , I dont speak of the successive peaks but the general form
Some days ago you see a descent and now it becomes flat but at a high level
I said that I was going to shut up, but UK has just revealed 521 extra deaths in care homes that are not included in official figures. Additonally :
HC One, which operates about 350 homes, said that as of 8pm on Monday there had been 311 deaths from confirmed or suspected Covid-19 with outbreaks in two thirds of its homes. MHA, a charitable operator, said there have been 210 deaths across 131 homes, with outbreaks in about half of its homes.
So if we assume a 10% mortality rate:
So we could estimate 5210 extra infected. Now lets guestimate that between 1/2 and 2/3 of the 430 000 people in 10 819 other care homes in the UK are also infected at a rate of 80%. That would give us anywhere between 176 000 and 234 000 extra cases.
Can you see where I'm going with this? Should we unilaterally add 5210 extra cases the the UK's figure?
If we do, why don't we add another 200 000 for all those other care homes.
Of course we wouldn't do this! None of the approximations I have made are justified by fact nor statistically valid. So why the F do we add similar guessed-at figures for France?
@Clumpton I read your comment about manual writing , are you evocating French issues in Marseille city ? If it is the case , in my knowledge , the issue was only there in Marseille , for the registry office of the city , who cannot transmit automaticaly the informations (covid or not, deaths or births) and only INSEE values are affected (delayed) , not hospitals ones
@JiPiBi Yes I was thinking of the problems in the Bouches de Rhône.
Whatever there will always be some small differences in reporting due to delays etc. Have you noticed the "Sunday Effect" on French cases? Nearly always a drop on Sunday and then an uptick on Monday.
FYI news about Confirmed and Possible in nursing homes seem to be detailled now
https://solidarites-sante.gouv.fr/IMG/pdf/point_de_situation_du_14_avril_2020.pdf
What is new for me is that all confirmed cases are included in the official confirmed counter and the Possible are counted separatly . So the official Confirmed value remains available.
Hi @CSSEGISandData, it seems like the correction announced here was introduced yesterday, but only for April 14th data and not for the past days.
As a result, the number of confirmed cases is surprisingly decreasing as of today, with a negative daily change as we can see on my dashboard using your data:
Do you plan on correcting also the data for the past days, or will it remain with this inconsistency?
edit: although the figures look different from the expected correction, so maybe the correction was already applied seamlessly and there is a different problem regarding April 14th data?
Thanks for tonight's correction!
Update 4/16: after communicating with solidarites-sante.gouv.fr, we decided to make these adjustments based on public available information. From April 4 to April 11, only "cas confirmés" are counted as confirmed cases in our dashboard. Starting from April 12, both "cas confirmés" and "cas possibles en ESMS" (probable cases from ESMS) are counted into confirmed cases in our dashboard.
Good news!
@CSSEGISandData I still see that the French Polynesia issue reported above was not solved: https://github.com/CSSEGISandData/COVID-19/issues/2094#issuecomment-611744632
@CSSEGISandData why count "cas probables en EHPAD et EMS" as confirmed cases when there is "cas confirmés en EHPAD et EMS" in the dashboard? this makes no sense!
As predicted, your data is useless now. Negative number of new cases, good job.
"useless" seriously? Do you realise how disrecpectful that is? how much time and dedication they have to invest to produce data of quality that we are all reusing? Of course there are occasional issues and they fix it progressively (the whole point in this thread was fixed already, the spike you're pointing is a different one...). But we're not talking about HMO here, this is a research lab, doing this freely. They have no responsibility to do anything whatsoever... If you're not happy with it, no one forces you to reuse this dataset, feel free to maintain your own (and understand the time and manpower it takes maybe...). Thanks again @CSSEGISandData!
@boogheta yes, useless. imagine calculating an increase rate with sinking data, which should by definition never happen. i don't have to respect anyone insisting on publishing unusable mislabeled data and i will proactively disrespect anyone not giving a fuck about deliberately misinforming the public.
Update 4/16: after communicating with solidarites-sante.gouv.fr, we decided to make these adjustments based on public available information. From April 4 to April 11, only "cas confirmés" are counted as confirmed cases in our dashboard. Starting from April 12, both "cas confirmés" and "cas possibles en ESMS" (probable cases from ESMS) are counted into confirmed cases in our dashboard.
Original: Translating press conferences and case reports invariably results in the loss of nuance and detail. In a daily briefing on April 3rd, the French health ministry director reported that there were 17,827 confirmed or suspected cases of COVID-19 in nursing homes. Data aggregation sites that rely on media reports to report case counts, such as ourselves, Worldometer, and BNO, followed the conclusion from this Reuters article that the nursing home cases were being reported separately from the confirmed case total and thus the two values should be summed to reach the true number of cases. This was also facilitated by the French Health Ministry reporting "confirmed cases" and "cases in social or medical-social establishments (nursing homes)" as two different quantities. After careful consideration and extensive research, we cannot confirm beyond a reasonable doubt that these two categories are mutually exclusive. For that reason, and in the interest of accuracy, we are reverting our numbers to reflect only those confirmed by the French Health Ministry and will be retroactively adjusting our time series data. We will continue to monitor the situation and will adjust our approach if a more appropriate way forward presents itself.