search

corona-warn-app / cwa-documentation

Project overview, general documentation, and white papers. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
3.28k stars 344 forks source link

Urgent Question on reduction of DKs distribution to 10 days #848

Closed Ein-Tim closed 2 years ago

Ein-Tim commented 2 years ago

Your Question

Since today, 07.02.2022 at ca. 22:00h, only the DKs of the last 10 days are distributed from the server to the app. What does this change mean: If a user had an increased risk encounter 12 days ago, will this encounter be deleted and the usual "Your risky encounter was deleted because it is older than 14 days now" message will be shown? Or in general: What happens to the risky encounters which are shown to the user and are older than 10 days? Will they be deleted or stay they until the ENF/ENS deletes them?

Cross ref.: https://github.com/corona-warn-app/cwa-server/pull/1711

thomasaugsten commented 2 years ago

The statement is not correct because only the key files of the last 10 days are distributed. Every key files includes keys from the last 14 days when the file is created. This means the user will still receive warnings in the last 14 days.

Ein-Tim commented 2 years ago

@thomasaugsten What exactly are the consequences of the change then?

thomasaugsten commented 2 years ago

Only 10 files are proceed by the app. If the user was for 10 days offline and trigger a download he will only receive the submitted keys from the submission of the last 10 days (warning from 10-24days). This means he will not receive some warning of the days 10 until 28.

thomasaugsten commented 2 years ago

@Ein-Tim can you please also correct the tweet regarding this topic

Ein-Tim commented 2 years ago

Ok but it is still possible to receive warning which are older than 10 days in the future?

thomasaugsten commented 2 years ago

Yes every key files includes warning from the day and 14 days in the past

Ein-Tim commented 2 years ago

Ok I will correct my Tweet, I'm sorry.

Could you maybe share a correct explanation what the change actually means for the user? (am besten auf Deutsch - das wäre super. Danke!)?

vaubaehn commented 2 years ago

@thomasaugsten I'm sorry, but I think it should be like this:

Only 10 files are proceed by the app. If the user was for 10 days offline and trigger a download he will only receive the submitted keys from the submission of the last 10 days (warning from 10-24days).

warning from days 1-24 days

This means he will not receive some warning of the days 10 until 28.

This means he will not receive some warning of the days 14 (or 15?) until 28.

Could you please confirm?

Ein-Tim commented 2 years ago

I will not tweet a clarification until I'm 100% certain what actually are the influences of this change.

"Die Änderung bedeutet nicht, dass es keine Warnungen mehr für Tage gibt, die mehr als 10 Tage in der Vergangenheit liegen. Die Änderung hat nur zur Folge, dass [Was? Werden Nutzende irgendeine Änderung bei den Warnungen spüren?]"

MikeMcC399 commented 2 years ago

@mlenkeit / @thomasaugsten

Please review the Solution Architecture document if changes need to be made, for instance in the Data Retention section which specifies 14 days.

Ein-Tim commented 2 years ago

@thomasaugsten

Does this sound like a good clarification to you: Die #CoronaWarnApp lädt seit heute Nacht nur noch Daten über Risikobegegnungen der letzten 24 Tage & nicht mehr die der letzten 28. Diese Änderung reduziert die Last auf den Endgeräten der #CWA-Nutzenden. Weiterhin werden nur Risikobegegnungen der letzten 14 Tage für Warnungen berücksichtigt.

However, the question why 28/24 days are downloaded will come up then.

thomasaugsten commented 2 years ago

Kann man so sagen wenn das nicht zu Verwirrung führt da dem Nutzer ja nur Warnungen der letzen 14 Tage angezeigt werden, ist die Frage ob man so ins Detail gehen sollte. Eine Keydatei enthält Warnungen für den Erstellungstag und 14 Tage in die Vergangenheit. Also die Keydatei von vor 10 Tagen enthält Warnungen vom 10. Tag bis zum 24. Tag.

Es gibt Randfälle wenn der Nutzer 10 Tage offline war, dass er nicht alle Warnungen der vergangenen 14 Tage erhält wenn diese Warnungen in den Paketen 11-14 waren.

Ein-Tim commented 2 years ago

Ok, also ist eher die beste Erklärung:

"Die Änderung wurde nur vorgenommen, um Last sowohl auf die Geräte der Nutzenden als auch auf die Server zu reduzieren. Nutzende erhalten auch in Zukunft Warnungen über die letzten 14 Tagen.

1/4

Was hat sich denn dann eigentlich geändert?

Die Corona-Warn-App lädt jeden Tag neue Pakete, in denen die Zufalls-IDs von positiv getesteten ("Warnungen") enthalten sind, herunter. In jedem dieser Pakete finden sich die Warnungen für den Tag an für den das Paket erstellt wurde & die Warnungen der letzten 14 Tage.

2/4

Bis gestern Abend wurden alle Pakete der letzten 14 Tage, die ja jeweils die Warnungen für den Erstellungstag und 14 Tage in die Vergangenheit beinhalten, heruntergeladen. Das wurde nun geändert und es werden nur noch die Pakete der letzten 10 Tage heruntergeladen.

3/4

Was bedeutet das für mich? Bekomme ich weniger Warnungen?

Ihr werdet von dieser Änderung nur mitbekommen, dass Risiko-Überprüfungen schneller laufen, und evtl. etwas weniger Speicherplatz auf eurem Gerät verwendet wird.

Auch weiterhin kriegt ihr Warnungen über einen Zeitraum von heute und den letzten 14 Tagen. Auch weiterhin kriegt ihr Warnungen über einen Zeitraum von heute und den letzten 14 Tagen.

4/4"

Feedback ist sehr Willkommen!

cc @vaubaen

vaubaehn commented 2 years ago

Hi @Ein-Tim , ich persönlich finde die Formulierungen sehr gut, v.a. sich nur auf die Nennung von "14 Tage" zu beschränken, ohne andere Zahlen ins Spiel zu bringen. Aus meiner ganz ego-zentristischen Perspektive wäre mein einziger Ergänzungsvorschlag zu 3/4:

Was bedeutet das für mich? Bekomme ich weniger Warnungen?

Ihr werdet von dieser Änderung nur mitbekommen, dass die Risiko-Überprüfungen schneller

~läuft~ laufen, und dass evtl. etwas weniger Speicherplatz auf eurem Gerät verwendet wird.

Auch weiterhin kriegt ihr Warnungen über einen Zeitraum von heute und den letzten 14 Tagen.

Thanks for your good work!

Ein-Tim commented 2 years ago

Thank you @vaubaehn! I'll change this!

@thomasaugsten could you say if this text is all correct?

Ein-Tim commented 2 years ago

@thomasaugsten

I'm sorry that I spread wrong information, could you please let me know if the text above is all right? This would be great & I will tweet it as clarification! Thank you!

thomasaugsten commented 2 years ago

Looks good for me

Ein-Tim commented 2 years ago

Thank you and sorry again.

Ein-Tim commented 2 years ago

Tweet's out: https://twitter.com/eintim2/status/1491073601164165122?s=21

mh- commented 2 years ago

@thomasaugsten Could you please explain why you chose to do this, and not limit the number of keys in new key files to 10 as well? Why would a 14-days-ago-meeting with someone who reported their infection status today (would still be detectable by CWA) be more relevant than a 14-days-ago-meeting with someone who reported their infection status 11 days ago (would now be excluded)?

mlenkeit commented 2 years ago

@thomasaugsten Could you please explain why you chose to do this, and not limit the number of keys in new key files to 10 as well? Why would a 14-days-ago-meeting with someone who reported their infection status today (would still be detectable by CWA) be more relevant than a 14-days-ago-meeting with someone who reported their infection status 11 days ago (would now be excluded)?

@mh- there are multiple reasons for that, but first and foremost, keys that are older than 10 days only make up ~2% of the keys in a package (at least since https://github.com/corona-warn-app/cwa-server/pull/1703).

For example, yesterday's key package for 2022-02-08 contains 358.105 keys, but only 5.811 keys (1.6%) are older than 10 days.

When we realized this, it became clear that there's very little leverage.

Additionally, the key packages are supposed to be immutable and the client has some optimizations built-in regarding that. If we for example took the key package t-5 (from 5 days ago) and regularly filtered out everything that is older than 10 days from now, the t-5 package would change everyday, and packages wouldn't be immutable anymore. Because of the low leverage we would have here anyway, we decided against doing such filtering at the moment.

Next, any client-side changes would take time to roll out, so we decided to do this server-side change and reduce the number of keys and files that are passed on ENF.

/fyi @Ein-Tim

mh- commented 2 years ago

Ok, but still - why not filter out keys older than 10 days server-side during upload from the clients?

thomasaugsten commented 2 years ago

@mh- The first step to reduce the key volume was to filter our TRL 1 and 2 this means a 14 days ago meeting is already filtered out. The step to publish only 10 key files helps to reduce the load on the ENF because we hand over all files to the ENF at each risk check.

mh- commented 2 years ago

@thomasaugsten

@mh- we hand over all files to the ENF at each risk check

Are you sure that's required and ENF isn't caching the old immutable files?

And still - filtering DKs older than 10 days already during upload, likely somewhere around here, would further reduce the load, wouldn't it?

thomasaugsten commented 2 years ago

This is an assumption but we have no proof or explanation how the caching is working at the moment. But we see a lot of failed calculation without a proper feedback of the ENF. Without a correct return value we cannot ensure the calculation was successful and the files are cached

It would reduce the load but it will only reduce the published files by 2% because the most DK older than 10 days are filtered out by the TRL 1-2 filter.

mh- commented 2 years ago

ok, thanks for the clarifications!