CSSEGISandData / COVID-19

Novel Coronavirus (COVID-19) Cases, provided by JHU CSSE
https://systems.jhu.edu/research/public-health/ncov/
29.1k stars 18.38k forks source link

About confirmed cases in France #2094

Closed CSSEGISandData closed 4 years ago

CSSEGISandData commented 4 years ago

Update 4/16: after communicating with solidarites-sante.gouv.fr, we decided to make these adjustments based on public available information. From April 4 to April 11, only "cas confirmés" are counted as confirmed cases in our dashboard. Starting from April 12, both "cas confirmés" and "cas possibles en ESMS" (probable cases from ESMS) are counted into confirmed cases in our dashboard.

Original: Translating press conferences and case reports invariably results in the loss of nuance and detail. In a daily briefing on April 3rd, the French health ministry director reported that there were 17,827 confirmed or suspected cases of COVID-19 in nursing homes. Data aggregation sites that rely on media reports to report case counts, such as ourselves, Worldometer, and BNO, followed the conclusion from this Reuters article that the nursing home cases were being reported separately from the confirmed case total and thus the two values should be summed to reach the true number of cases. This was also facilitated by the French Health Ministry reporting "confirmed cases" and "cases in social or medical-social establishments (nursing homes)" as two different quantities. After careful consideration and extensive research, we cannot confirm beyond a reasonable doubt that these two categories are mutually exclusive. For that reason, and in the interest of accuracy, we are reverting our numbers to reflect only those confirmed by the French Health Ministry and will be retroactively adjusting our time series data. We will continue to monitor the situation and will adjust our approach if a more appropriate way forward presents itself.

cipriancraciun commented 4 years ago

@CSSEGISandData also regarding invalid numbers for France, it appears that the daily reports for 03-23-2020.csv for French Polynesia includes also the values for the France mainland. (The series files seems to be correct.)

For example:

I.e. for 23rd the France row is missing, and the numbers for French Polynesia jump drastically from 22nd, but then go back again for 24th.

ivandebono commented 4 years ago

This is the data source you should be using: https://www.gouvernement.fr/info-coronavirus/carte-et-donnees

greg-minshall commented 4 years ago

also (if this is better than opening a new case... :), the 04-09-2020.csv daily file has an odd French entry:

00250,,,France,2020-04-09 23:02:19,46.2276,2.2137,117749,12210,23206,82333,",,France"

(that leading "00250" seems bogus.)

yetzt commented 4 years ago

you're back to the wrong number: Screenshot

yetzt commented 4 years ago

meme

alfkoehn commented 4 years ago

Just to have the correct (i.e. official from other sources) numbers here:

The data here says: 117749 Number in other data sources [1,2]: 86334

[1] https://dashboard.covid19.data.gouv.fr/ [2] https://github.com/opencovid19-fr/data/blob/master/ministere-sante/2020-04-09.yaml

boogheta commented 4 years ago

If France were releasing confirmed data with the same accuracy they do for hospital cases, things would be so much simplier, but hey, that's how we do in France with official data, never simple... Thanks for your great work @CSSEGISandData, it must not be easy dealing with the different way of reporting data in every different country!

JiPiBi commented 4 years ago

@boogheta Perhaps as french yourself, could you explain the situation of nursing homes, instead of inappropriate comment.

boogheta commented 4 years ago

@JiPiBi I'm not sure how this is inappropriate. France has a long history of cultural lack of transparency and releasing hardly valid data (see for instance companies register data...). The lack of transparency from the government on nurses home situation makes it hard to explain properly, but basically people are not all tested, and as soon as a spread is observed in a nursing home, all sick people in this home are accounted. This does not legitimate government not publishing detailed figures coming from these since they do account them in their total national confirmed cases and in a separate national deaths count.

ivandebono commented 4 years ago

The lack of transparency from the government on nurses home situation makes it hard to explain properly, but basically people are not all tested, and as soon as a spread is observed in a nursing home, all sick people in this home are accounted. This does not legitimate government not publishing detailed figures coming from these since they do account them in their total national confirmed cases and in a separate national deaths count.

Do you mean that "cas confirmés" is over-reported here: https://www.gouvernement.fr/info-coronavirus/carte-et-donnees ?

boogheta commented 4 years ago

I wouldn't go as far as "over-reported", but according to the way it is accounted in nurses home and what the government announced, it should rather be called "cas confirmés ou fortement suspectés" yes.

JiPiBi commented 4 years ago

@boogheta I disagree, the cases in nursing homes are separatedly presented on the official dashboard and not aggregate in the national confirmed cases for the reasons you gave. Secondly the people from ESMS who died in hospitals are in the hospital death counter and the people died outside hospital are in the death global counter.

boogheta commented 4 years ago

@JiPiBi No, according to official speeches, the cases of deceased are presented separately only, not the cases of confirmed. This is coherent with the fact that hospitals do not report data on confirmed: deceased must be added from both sources (hospitals and ehpad), whereas there is no breakdown for confirmed (neither by source nor by region), only global figures accounted in different ways and which might overlap... This is exactly the point @CSSEGISandData is explaining in this issue and why they are reverting (tonight?) to the good value which is around 90 000 and not 110 000 as was reported over the past week but not touching the number of deceased which is the good one. You can read more about it in French here: https://www.liberation.fr/checknews/2020/04/05/covid-19-pourquoi-des-sites-evoquent-90-000-cas-en-france-contre-68-000-au-bilan-officiel_1784232

JiPiBi commented 4 years ago

@boogheta No about what ?? What did I say that is not coherent with the following yesterday's dashboard

image

yetzt commented 4 years ago

As far as i understand it, the ESMS/EHPAD cases are not in fact confirmed cases, but consist in fact in part of speculative cases. therefore adding them to a data field named Confirmed is deeply problematic. If there is no valid conclusive official count of definitely confirmed cases outside the official number, there is nothing that can be added without setting fire to any meaning.

Those 31415 unconfirmed cases do by definition include an undisclosed amount of false positive cases. Including these in the total of confirmed cases does noet adhere to any standard, scientific, journalistic or otherwise.

The ever-changing ad-hoc-reasoning by JHU CSSE is deeply worrying.

JiPiBi commented 4 years ago

@yetzt You are right , the 31415 are not for the moment sorted between confirmed and suspected , a ratio of 1/3 of confimed has been annouced yesterday and we are waiting on more accurate values In France there are about 7000 nursing homes , which where not in the informatic network of health system . This is the explanation of huge increase in death cases because many of these old people died very quickly in the nursing homes and not in the hospitals and the true values were not known by health system, only by city services . It seems that the people could be in quite good form and some hours later became very sick and die People after a certain age can't support ICU even if there are in an hospital. Imagine to be 2 or 3 weeks unconscious on the belly . Even for young people it is hard to recover, it may last weeks or moths . In hospital about 25-30% of people under 80's are in ICU . Above 80's only about 10% are in ICU . And they represent the main part of deaths ...

JiPiBi commented 4 years ago

Update

image

ivandebono commented 4 years ago

I have another question: What is the total number of recovered? "Retours à domicile" means patients released from hospital. What about those who had the virus but were never hospitalised? Do we just use this? recovered = total infected - total dead - active cases

Can we find a figure for total active cases somewhere?

JiPiBi commented 4 years ago

Recovered here is clearly recovered after being hospitalyzed. People who had mild symptoms were asked to stay at home : mainly not tested, in tight contact with their doctor to see if the sickness is evoluting.
In the family , there is also a possibility that another person is positive but these values are not known and if they recover at home these values are also not known and probably will never been known. So as we dont know how many people where infected , the notion of recovered means nothing for the moment . Only serologic tests could give a real idea of the situation .

See on worldometers the informations about this experiment in Germany :

A study by the University of Bonn has tested a randomized sample of 1,000 residents of the town of Gangelt (an epicenter of the outbreak in Germany) and found that 2% of the population was currently infected and 14% were carrying antibodies suggesting that they had already been infected — whether or not they experienced any symptoms. Eliminating an overlap between the two groups, the team concluded that 15% of the town have been infected with the virus [source]

If these findings are correct, Germany’s actual death rate could be as low as 0.22% (2,607 deaths / (2,607 cases that have resulted in death + 1,172,000 cases that have resulted in recovery)). Assuming 14% of the German population of 83,700,000 (1,172,000 people) have been infected and have recovered.

Source : https://spectator.us/covid-antibody-test-german-town-shows-15-percent-infection-rate/

ivandebono commented 4 years ago

The notion of recovered is very clear. There are many countries in Europe where people who tested positive and had mild symptoms were made to self-isolate at home, then tested again periodically, and finally declared recovered when they tested negative (the Prince of Wales is one example). They were then added to the tally of recovered cases.

If France only considers as recovered those patients who were released from hospital, it is under-reporting recoveries.

JiPiBi commented 4 years ago

You are right. But Prince of Wales is not the good example. John Doe on SmallCity has not the same attention :-)

People that recovered alone at home have been rarely tested positive, because lack of tests at the beginning. I m not justifying, I am explaining the values.

Clumpton commented 4 years ago

Just to make it 100% clear. Here in France they are currently mainly testing people admitted to hospital with advanced symptoms, front-line medical staff (plus politicians etc.). The general public are not being tested even if symptomatic. If a nursing home resident falls ill, is hospitalized and tests positive then the whole home is classed as 'possible'. The "recovered" cannot include people who self-isolated and got well since they were never confirmed in the first place (French logic). The govt. have announced the start of 'massive' testing in nursing homes, so those 'possibles' will gradually turn into 'confirmed' - this means that we should not include them in the statistics at the moment as they will eventually end up being counted twice! The French count includes ALL confirmed cases as per this graph (from https://www.gouvernement.fr/info-coronavirus/carte-et-donnees at 01:00 CEST 2020-04-11) canvas With this key key

Translated it reads (from top to bottom)

I hope that helps clarify the matter

ivandebono commented 4 years ago

Thank you.

The different reporting methods make forecasting harder. I would have expected standardised reporting at least across the EU, or what remains of it.

JiPiBi commented 4 years ago

@Clumpton Nothing to add to your explanations.

Remarks:

JiPiBi commented 4 years ago

@ivandebono Agree with the reporting differences making it hard to compare. For example, knowing new ICU and new hosp is very important now in France and Europe. At the beginning cases and deaths are important, in the middle, the exponential increase and quantity of beds and ICU at the peek the new hosp and new ICU, and after I suppose all the dead includind surmortality by class of age and the percentage of infected people

I think WHO should now propose common reporting.

Clumpton commented 4 years ago

@JiPiBi

* Add décès in hosp+ recovered+ currently hosp and compare to 65420. I dont understand the result.

I don't understand where you get 65240 from?

JiPiBi commented 4 years ago

Look above, I put yesterday the most recent informations. There are 6 colored cells from SPF. My purpose is to understand the value that counts all people that went to hospital in the upper middle cell.

Clumpton commented 4 years ago

@JiPiBi OK I see now, that 65240 cell is the total over the pandemic, if you look at the 3rd line it says "dont 31267 en cours" (of which 31267 are currently in hospital). Add 32167 (current), 24932 (number discharged) and 8598 (hospital deaths) you get 64797 - a disparity of 443 - perhaps those are cases that were hospitalised and tested negative?

JiPiBi commented 4 years ago

You got my issue to understand this gap, except that the current value for hosp is 31267(not 32167) and the difference is 623 persons. It was almost the same 2 days ago. There is an hypothesis that some of them could be linked to people sent to other regions, because it seems that they are counted again as new for the new hospital. Question asked without answer for the moment.

Clumpton commented 4 years ago

Oops sorry about that slip. The hypothesis of the hospital changes counting twice sounds quite likely. On second thoughts, maybe it is due to deaths sometimes being reported 1-2 days later than hospitalizations? For example there have been several hospitals that have had to resort to pen & paper records recently due to cyber-attacks.

ivandebono commented 4 years ago

The "recovered" cannot include people who self-isolated and got well since they were never confirmed in the first place (French logic).

Can you confirm, then, that everyone who tested positive was admitted to hospital.

And if you include "possible" cases with the grand total of infections, why not include "possible" cases with the grand total of recoveries? In statistics we call it systematic bias.

Clumpton commented 4 years ago

@ivandebono

Can you confirm, then, that everyone who tested positive was admitted to hospital.

Of course not, many have self-isolated, but SO FAR in France the huge majority of tests have been done on people exhibiting symptoms that prompted hospitalization. I am fully aware that the actual situation in France is forcibly far worse than indicated by the statistics due to insufficient testing. The grand total of recoveries is (as stated above, and as far as I know) those who have been discharged from hospital. I suspect that the govt. doesn't currently have the faintest idea of how many have recovered in nursing homes or in self-isolation, so there are no possible cases to include in recoveries. So the systematic bias exists only if you insist on including a figure for "possible" cases in the confirmed cases while being unable to include or estimate the "recovered possible cases" in recoveries.

Please remember that these "possible" cases are a number pulled out of the air by a minister. Probably (N° of Homes with a case) (average occupants in homes) (90% infection rate).

Anyway if Johns Hopkins wishes to insist that France has 133k while the French govt. says 95k they will only lose all credibility in France, all while increasing the paranoia that the French people have about being lied by their govt. That's me finished.

JiPiBi commented 4 years ago

Do you really think that the number of confirmed or recovered is so important now, you dont really know both values... Even deaths are unknown, home dead people covid suspected are not tested... You will only measure later surmortality...

IMHO what is more important now is the number of hospitalyzed in or out ICU, and the evolution of these values, day after day.

Clumpton commented 4 years ago

@JiPiBi

I agree I don't know why people are wittering about possible recovered etc.

ivandebono commented 4 years ago

Of course it's important. When you try to fit epidemiological models to the data, you want your data to be as accurate as possible. That's basic statistical science.

If you're making policy based on the fitted models, it becomes even more vital.

JiPiBi commented 4 years ago

But when data are not available, what can you do ?

Do you really think that chinese data were faithfull and helped other countries to prepare themselves? At first , people said, its China, they have strange habits with animals and the epidemic kills only old and already very sick people. Comorbidities as they said.

After, it's the turn of Italy, people say , they have no modern hospital and they have so many old people and olds live in the same house than youngs. Italian are also so latin... But Lombardia is not a third world region... And after in France you meet difficulties with a region, you discover that you have not enough masks, not enough tests, and people in nursing homes are not well protected.

It seems that countries never learn from the previous infected ones if you look at the situation in the US. They only know to apply confinment and they add economic troubles to health troubles.

In Japan , there is no confinment, there are no many tests , the cities are crowdy and they have few deaths.

Italy : many tests, many deaths Germany: many tests, few deaths Japan: few tests, few deaths.

Is there a model fitting all that situations ?

Moreover, what seems important in these epidemic is local red hot situations : every country has its Wuhan and its Hubei. In France we have 3 or 4 regions with difficulties. In Italy, there is Lombardia and Bergamo and they dont represent the whole country. So you need to be more specific than the country scale.

And now apart from statistics, there is a simple fact , how do you deal with a huge arrival of very sick people in the same hospital, you are in a scale of time where every hour is important ?

So I dont see for the moment how data can help to deal with such a crisis, if you are not prepared culturally at first and well equiped in second .

But strangely , small countries like Portugal, Croatia , Austria have very good results. Can data help to understand all that diversity and make usefull forecasts ? Some days ago, we saw that the results were better in Italy. But even Italian descent is quite chaotic. Could data explain that?

JiPiBi commented 4 years ago

Pls , Look at the new hosp evolution in Italia.(that is new people hospitalyzed every day) Can model show the real tendancy , I dont speak of the successive peaks but the general form Some days ago you see a descent and now it becomes flat but at a high level image

Clumpton commented 4 years ago

I said that I was going to shut up, but UK has just revealed 521 extra deaths in care homes that are not included in official figures. Additonally :

HC One, which operates about 350 homes, said that as of 8pm on Monday there had been 311 deaths from confirmed or suspected Covid-19 with outbreaks in two thirds of its homes. MHA, a charitable operator, said there have been 210 deaths across 131 homes, with outbreaks in about half of its homes.

So if we assume a 10% mortality rate:

So we could estimate 5210 extra infected. Now lets guestimate that between 1/2 and 2/3 of the 430 000 people in 10 819 other care homes in the UK are also infected at a rate of 80%. That would give us anywhere between 176 000 and 234 000 extra cases.

Can you see where I'm going with this? Should we unilaterally add 5210 extra cases the the UK's figure?

If we do, why don't we add another 200 000 for all those other care homes.

Of course we wouldn't do this! None of the approximations I have made are justified by fact nor statistically valid. So why the F do we add similar guessed-at figures for France?

JiPiBi commented 4 years ago

@Clumpton I read your comment about manual writing , are you evocating French issues in Marseille city ? If it is the case , in my knowledge , the issue was only there in Marseille , for the registry office of the city , who cannot transmit automaticaly the informations (covid or not, deaths or births) and only INSEE values are affected (delayed) , not hospitals ones

Clumpton commented 4 years ago

@JiPiBi Yes I was thinking of the problems in the Bouches de Rhône.

Whatever there will always be some small differences in reporting due to delays etc. Have you noticed the "Sunday Effect" on French cases? Nearly always a drop on Sunday and then an uptick on Monday.

JiPiBi commented 4 years ago

FYI news about Confirmed and Possible in nursing homes seem to be detailled now

https://solidarites-sante.gouv.fr/IMG/pdf/point_de_situation_du_14_avril_2020.pdf

What is new for me is that all confirmed cases are included in the official confirmed counter and the Possible are counted separatly . So the official Confirmed value remains available.

boogheta commented 4 years ago

Hi @CSSEGISandData, it seems like the correction announced here was introduced yesterday, but only for April 14th data and not for the past days.

As a result, the number of confirmed cases is surprisingly decreasing as of today, with a negative daily change as we can see on my dashboard using your data:

france_april14 france_april14_2

Do you plan on correcting also the data for the past days, or will it remain with this inconsistency?

edit: although the figures look different from the expected correction, so maybe the correction was already applied seamlessly and there is a different problem regarding April 14th data?

boogheta commented 4 years ago

Thanks for tonight's correction!

CSSEGISandData commented 4 years ago

Update 4/16: after communicating with solidarites-sante.gouv.fr, we decided to make these adjustments based on public available information. From April 4 to April 11, only "cas confirmés" are counted as confirmed cases in our dashboard. Starting from April 12, both "cas confirmés" and "cas possibles en ESMS" (probable cases from ESMS) are counted into confirmed cases in our dashboard.

Clumpton commented 4 years ago

Good news!

cipriancraciun commented 4 years ago

@CSSEGISandData I still see that the French Polynesia issue reported above was not solved: https://github.com/CSSEGISandData/COVID-19/issues/2094#issuecomment-611744632

yetzt commented 4 years ago

@CSSEGISandData why count "cas probables en EHPAD et EMS" as confirmed cases when there is "cas confirmés en EHPAD et EMS" in the dashboard? this makes no sense!

yetzt commented 4 years ago

As predicted, your data is useless now. Negative number of new cases, good job.

Screenshot

boogheta commented 4 years ago

"useless" seriously? Do you realise how disrecpectful that is? how much time and dedication they have to invest to produce data of quality that we are all reusing? Of course there are occasional issues and they fix it progressively (the whole point in this thread was fixed already, the spike you're pointing is a different one...). But we're not talking about HMO here, this is a research lab, doing this freely. They have no responsibility to do anything whatsoever... If you're not happy with it, no one forces you to reuse this dataset, feel free to maintain your own (and understand the time and manpower it takes maybe...). Thanks again @CSSEGISandData!

yetzt commented 4 years ago

@boogheta yes, useless. imagine calculating an increase rate with sinking data, which should by definition never happen. i don't have to respect anyone insisting on publishing unusable mislabeled data and i will proactively disrespect anyone not giving a fuck about deliberately misinforming the public.