search

corona-warn-app / cwa-documentation

Project overview, general documentation, and white papers. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
3.28k stars 345 forks source link

R-Wert not in line with Github repo nowcast #872

Closed btreut closed 2 years ago

btreut commented 2 years ago

the R-Wert and its tendency displayed in CWA Android is different from what I find in https://github.com/robert-koch-institut/SARS-CoV-2-Nowcasting_und_-R-Schaetzung/blob/main/Nowcast_R_aktuell.csv

Why?

Ein-Tim commented 2 years ago

The R-Wert for today is in line with the one from the RKI. The R-Wert from the RKI can be found here: https://www.bundesregierung.de/breg-de/themen/coronavirus/fallzahlen-coronavirus-1738210

MikeMcC399 commented 2 years ago

@btreut

You didn't actually say what value for the 7-Day R Value which you are seeing in the app. I am seeing 0.99 today.

You can see on the CWA web Dashboard if you scroll to the bottom and select RKI: Covid-19 Trends (GER) that the value displayed there is also 0,99.

You can find the same value in the daily report from RKI https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Situationsberichte/Maerz_2022/2022-03-17-de.pdf?__blob=publicationFile from yesterday.

image

If you want to find out about the correlation of the dates used in the table on https://github.com/robert-koch-institut/SARS-CoV-2-Nowcasting_und_-R-Schaetzung/blob/main/Nowcast_R_aktuell.csv compared to what is shown in RKI's daily report, you should probably ask in the https://github.com/robert-koch-institut/SARS-CoV-2-Nowcasting_und_-R-Schaetzung repository.

The value in the app today seems to be correct.

vaubaehn commented 2 years ago

@btreut Additionally to the nice explanation from @MikeMcC399 : The values in the csv you linked to are not up to date, the latest values are for 2022-03-14. Also, when you look to the history of that repository, you will see that older dates are updated/corrected: https://github.com/robert-koch-institut/SARS-CoV-2-Nowcasting_und_-R-Schaetzung/commits/main/Nowcast_R_aktuell.csv Assumingly, this is due to "Nachmeldungen" from the federal states that result in higher r-values calculated for the past. But like Mike wrote, it's better to directly ask in the RKI repo.

btreut commented 2 years ago

I see in cwa 0.99 and increasing tendency. The tendency in the actual now casting table is decreasing. The latest R-Wert in the now casting table is 1 for 14.03.2022 and it cannot be calculated for today.

MikeMcC399 commented 2 years ago

@btreut

I see in cwa 0.99 and increasing tendency. The tendency in the actual now casting table is decreasing. The latest R-Wert in the now casting table is 1. It cannot be calculated for today.

Tap the ℹ️ icon on the statistics tile to read the help about trend calculation.

The value displayed yesterday was 0.98 taken from the daily report of March 16, 2022 https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Situationsberichte/Maerz_2022/2022-03-16-de.pdf?__blob=publicationFile.

0.98 to 0.99 is calculated as a 1% trend increase which is shown as a red up-arrow today.

I don't believe that you can use the table which you are referring to in order to check the accuracy of the data displayed in the app. Edit: You can use the GitHub data as reference, but you need to refer to multiple tables. See https://github.com/corona-warn-app/cwa-documentation/issues/872#issuecomment-1073284877.

Based on referring to RKI's daily reports I cannot see any error in the data displayed in the app.

btreut commented 2 years ago

Sorry, conflicting numbers on different pages of the same source (RKI) do not increase the confidence in the presented numbers. The figures 4 pasted by @MikeMcC399 above clearly show a decreasing tendency, but cwa displayed an increasing.

MikeMcC399 commented 2 years ago

@btreut

I agree that the trend shown in the app contradicts the graph. The "7-Day R Value" is a difficult statistical value to understand and use.

You might find the explanations and links in

helpful for more background reading.

Ein-Tim commented 2 years ago

@btreut Was it your intention to close this issue?

btreut commented 2 years ago

my arguments are more or less wiped away and labelled as irrelevant. I do not want to get lost in discussions about other data sources. To my knowledge the github repo for nowcasting is the primary source and if this is not acknowledged then I do not want to get lost in strange discussions about other publications of RKI, with different numbers.

vaubaehn commented 2 years ago

my arguments are more or less wiped away and labelled as irrelevant. I do not want to get lost in discussions about other data sources. To my knowledge the github repo for nowcasting is the primary source and if this is not acknowledged then I do not want to get lost in strange discussions about other publications of RKI, with different numbers.

@btreut No one wants to wipe away your arguments. What we were trying to do is to find an explanation together with you. You can be sure that the community here has a critical eye on functioning and possible wrong data sources, that I may certainly speak for everyone participating in this discussion, that we're thankful for everyone coming up with such an issue. However, I'd like you to again have a look at the RKI data set in the corresponding repo. From the description it's obviously a different dataset aimed for research. According to the history of commits to the dataset from their maintainers, it looks like it's trimmed for accuracy and changed whenever new data are coming in ('Nachmeldungen'). The r-value that is reported daily (and that is displayed in the app and on the RKI website) can only be weak estimate with regard to the reporting processes and capacities in the federal states.

MikeMcC399 commented 2 years ago

@btreut

my arguments are more or less wiped away and labelled as irrelevant.

I am sorry if you feel that you were not taken seriously! @vaubaehn has written in parallel to me and also says the same.

I took another look at the tables and it seems that the values which are published in RKI's daily situation reports are not always aligned to the data in the table. That may be due to the daily recalculations which also affects previous days. See Daten and Datenaufbereitung

"Die Schätzwerte zur Anzahl von Neuerkrankungen und der R-Schätzung zu früheren Tagen können von den Angaben in früheren Versionen der Tabelle abweichen, weil täglich der Gesamtverlauf, aufgrund der aktuell verfügbaren Daten, neu geschätzt wird.".

The app just displays the data which RKI provides on their data server, so I'm afraid that you won't get any more detailed insight and feedback here in the Android app repository. This part of the data connection from the app back to the data server is not open-source, so we from the community can't see what is being done. We do know though that the data publication involves a daily manual approval process by RKI before the data is passed on from RKI to the app.

The readme section Administrative und organisatorische Angaben says: "Inhaltliche Fragen bezüglich des Nowcastings und der R-Schätzungen können an an das RKI unter info@rki.de gestellt werden." It sounds like you do not want to go any further with your issue, but if you do, then the RKI e-mail address may be able to give you an answer.

MikeMcC399 commented 2 years ago

@dsarkar Please move this issue to the cwa-documentation repository, since it is an issue affecting both Android and iOS. I will research the issue in more detail and (hopefully) provide a better answer.

MikeMcC399 commented 2 years ago

@btreut

After researching the data on https://github.com/robert-koch-institut/SARS-CoV-2-Nowcasting_und_-R-Schaetzung/blob/main/Nowcast_R_aktuell.csv it is clear the the differences in trends which we see in the CWA app compared to looking at a single file on GitHub are due to recalculation.

For the last 5 weekdays the app displayed the following values taken from the daily reports on https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Situationsberichte/Gesamt.html?nn=13490888

RKI Daily report date 7-Day R-value Cases up to
14.03.2022 1,00 10.03.2022
15.03.2022 0,99 11.03.2022
16.03.2022 0,98 12.03.2022
17.03.2022 0,99 13.03.2022
18.03.2022 1,00 14.03.2022

You can find exactly these values in https://github.com/robert-koch-institut/SARS-CoV-2-Nowcasting_und_-R-Schaetzung/blob/main/Nowcast_R_aktuell.csv however, you need to take the data from each daily archive file, not from the latest file. CWA displays the best information it has each day. It does not replace the historical data for previous days when it is calculating trends though.

GitHub file date PS_7_Tage_R_Wert for day
14.03.2022 1,00 09.03.2022
15.03.2022 0,99 10.03.2022
16.03.2022 0,98 11.03.2022
17.03.2022 0,99 12.03.2022
18.03.2022 1,00 13.03.2022

The recalculated previous days in the file for 18.03.2022 have different values, and therefore the trend is not the same as the trend shown in the app.

GitHub file date PS_7_Tage_R_Wert for day
18.03.2022 1,10 09.03.2022
18.03.2022 1,09 10.03.2022
18.03.2022 1,06 11.03.2022
18.03.2022 1,03 12.03.2022
18.03.2022 1,00 13.03.2022

In summary, the 7-Day R Value data and trends shown in the Corona-Warn-App are correct. In order to reconcile the data displayed in the app with the data on GitHub it is necessary to use multiple .csv files from the Archiv folder.

Thank you for raising this issue. I hope that your question is now finally answered!

PS There is a typo in the subject of this issue. It should say "github" not "guthub". You might like to correct this just in case somebody searches using the title. (corrected)

MikeMcC399 commented 2 years ago

It may be worth requesting RKI to provide a single table showing all the historical values of the "7-Day R Value" which have been published to the daily situation report and shown in the Corona-Warn-App. If somebody wants to make that request it should probably go through SARS-CoV-2-Nowcastingund-R-Schaetzung/issues or the contacts listed in SARS-CoV-2-Nowcasting und -R-Schätzung.

Without such a table it will always be necessary to access multiple archived files to retrieve the historical values which have been shown daily in the statistics tiles of the Corona-Warn-App.

I'm not planning to make this request personally as I do not have this need.

btreut commented 2 years ago

@MikeMcC399 thanks for following up the topic. Yes, I know that the values change in time, but these also get more precise and the confidence range narrows. I know that I have to look at the archived older versions for getting the published number. I had previously a similar discussion with the guys at RKI (quite some time ago) about that topic and after that the values displayed in CWA were some time in line with the ones displayed in the nowcasting github repo. And yes, I felt that I am not taken seriously. When I started this issue, I had the impression that the numbers were again diverging.

MikeMcC399 commented 2 years ago

@btreut Thanks also for responding and you are welcome to the follow up. I'm really sorry if you felt you weren't taken seriously!

Probably the easiest place to view the "7-Day R Value" / "7-Tage-R-Wert" and trends is in the RKI dashboard Covid-19-Trends in Deutschland im Überblick. I noticed that it gets updated Monday - Friday, not at the weekend. According to the help for this data the difference and trend are calculated based on the previous week's value.

image

The trend arrow in CWA only compares to the previous day, so you can't really judge if it is an on-going trend or just a one-day exception. When the statistic tile was added to the Corona-Warn-App quite a long time ago, the dashboard I mentioned above was not available. Now the dashboard is available, I would prefer it personally.

If you find any bugs in the data in the future, then please open a new issue in this cwa-documenation repository or if you want to suggest an improvement then cwa-wishlist is the right place. Of course if you already have contacts to RKI, you might want to use those as well.