Unnecessary delay in exposure notification due to delayed fetching of Diagnosis Keys?

[kbobrowski]

Current API allows for querying days on which Diagnosis Keys became available:

$ date +%Y-%m-%d
2020-06-13

$ curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date
["2020-06-09","2020-06-10","2020-06-11","2020-06-12"]

Apps use this endpoint to check if these dates are already in cache, and if not to download Diagnosis Keys for missing days using endpoint:

$ curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-12 --output diagnosis_keys_2020-06-12.zip

Let's assume a scenario where someone had contact with infected person. Infected person uploads Diagnosis Keys on 2020-06-12, but these Diagnosis Keys are bundled into package that is only available to download on 2020-06-13. This introduces a day of delay, during which a person who had contact with diagnosed person is unaware of this contact and is not able to self-isolate, potentially endangering people around.

Interestingly it seems to be possible to get Diagnosis Keys uploaded on current day, but only using hour API endpoint:

$ curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-13     
<!-- error -->

$ curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-13/hour
[3,8,10,12]

$ curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-13/hour/12 --output diagnosis_keys_2020-06-13_12.zip

This endpoint seems to be only used in a function which is marked to be dropped before release. Functionality to fetch hours for current day existed, but it was removed in this pull request.

Is there a reason not to continuously download Diagnosis Keys which are being made available during current day, and instead wait until package for complete day is ready?

As far as I understand this would not introduce any privacy issues since they are already handled by upload mechanism documented here, and would just let people know faster that they were exposed. Privacy of this solution can also be confirmed by looking at latest Diagnosis Key in each bundle, e.g. in a bundle from current day (2020-06-13, hour 12) latest Diagnosis Key has interval number 2653200, meaning it was generating RPIs yesterday:

$ TZ="utc" date -d@$((2653200 * 600)) +"%Y-%m-%d %H:%M"
2020-06-12 00:00

---

Internal Tracking ID: EXPOSUREAPP-1567

[nilsalex]

Wonder why there are no Diagnosis Keys published... Let's say 500 people a day catch covid, so there is daily probability of p=0.00000624 that each individual person will catch it. But with 10.000.000 users the probability that 1 of those users will get it is touching 100%. With only 100.000 users it would be already 60%.

I wonder if there is a way to tell if cwa-server-1.0.8 (with key padding) is already live. Because if it is, we should really see some keys in the near future.

[christian-kirschnick]

I wonder if there is a way to tell if cwa-server-1.0.8 (with key padding) is already live. Because if it is, we should really see some keys in the near future.

1.0.8 is not yet live - it is currently in our staging environments and in testing.

Wonder why there are no Diagnosis Keys published... Let's say 500 people a day catch covid, so there is daily probability of > p=0.00000624 that each individual person will catch it. But with 10.000.000 users the probability that 1 of those users will get it is touching 100%. With only 100.000 users it would be already 60%.

I'm also wondering about this. With a 10M user base and ~600 positive tested people in Germany each day, statistically, we should already see daily files even w/o any adjustments. The only reason I see is that CWA is not yet that distributed in the Corona hotspots yet.

[cfritzsche]

I wonder if there is a way to tell if cwa-server-1.0.8 (with key padding) is already live. Because if it is,

we should really see some keys in the near future.

1.0.8 is not yet live - it is currently in our staging environments and in testing.

Wonder why there are no Diagnosis Keys published... Let's say 500 people a day catch covid, so there is daily probability of > p=0.00000624 that each individual person will catch it. But with 10.000.000 users the probability that 1 of those users will get it is touching 100%. With only 100.000 users it would be already 60%.

I'm also wondering about this. With a 10M user base and ~600 positive tested people in Germany each day, statistically, we should already see daily files even w/o any adjustments. The only reason I see is that CWA is not yet that distributed in the Corona hotspots yet.

Could the operator find out and disclose how many keys have been submitted yet below the threshold? Or how often a teleTAN was requested? Would be interesting if it’s more than 0.

[christian-kirschnick]

1.0.8 is now live.

[SebastianWolf-SAP]

Hi everbody,

as cwa-server 1.0.8 is now live (thanks for the notification @christian-kirschnick), I'd like to come back to the original topic of this thread and also fulfill my promise which I made over a week ago. Please apologize the late answer, the last days were nothing less than overwhelming and we still need to recover. ;)

But back to topic: We sat together with the mobile colleagues and compiled a little bit of more information in addition to some hints which were already given by @christian-kirschnick, here we go:

tl;dr

The initial feedback I gave you in https://github.com/corona-warn-app/cwa-documentation/issues/236#issuecomment-643762518 was more or less correct. For reasons of consistency between the apps and because of API restrictions, we are basically restricted to the 24h interval on both platforms. But: Improvements are in the backlog and they will be tackled as soon as possible.

Long Story

The information which @kbobrowski received from Google is of course correct. The Google Framework - in theory - supports 'provideDiagnosisKeys' with more than 1 file per call. However, one requirement is that a batch number and the batch size that is adapted to the file amount sent to the method. Of course, the batch number would then be incremented with each file. On the other side, the Apple Framework has a hard limit on 15 files and the exposure detection there does not support batching (as of now). This can be seen in the reference documentation (https://developer.apple.com/documentation/exposurenotification/setting_up_an_exposure_notification_server?language=objc#see-also). The file limitation is derived from the 'detectExposuresWithConfiguration' method (https://developer.apple.com/documentation/exposurenotification/enmanager/3586331-detectexposureswithconfiguration?language=objc).

On the app side both currently only support fetching diagnosis key packages based on the date index. Their implementation details differ in terms of background job processing, scheduling and index fetching, but - and that's important - the functionality is scope-wise equivalent and we also want and need to keep them equivalent. To explain the individual characters of the app:

• the iOS App queries the hour packages, but does not use them for the exposure detection
• the Android App queries only date packages, so it does not use the hour index at all
• the iOS App schedules a job every 2 hours to check if a exposure detection is necessary
• the Android App has a 24-hour job that schedules a job based on a random time in a set interval that checks if exposure detection is necessary (load balancing!)
• both Apps have a behavior that allows only daily exposure detection updates
• the iOS App could in theory support 15 detections per day if all keys are provided in one file
• the Android App could in theory support 20 detections per day if the API token is persisted for that day and batching is introduced

Solving this issue without Apple/Google enhancements would require some changes not only in the apps, but also in the cwa-server backend. As mentioned in the tl;dr section: Improvements here have made it to the backlog and we simply need to see how fast we can move here.

Moreover, we are continuously evaluating how the infrastructure works - that will most certainly also include the daily download, the follow-up calculations afterwards, possible delays in notifications because of that etc. You've seen the aforementioned cwa-server 1.0.8 with the key padding and cwa-app-android 1.0.4 with an improved key download mechanism in place. So, we are listening and will continue to do so. However, changes are not that easy as we need consistency over both client platforms, it needs to be tested thoroughly and finally everything also needs to be accepted by all stakeholders.

Therefore, we will keep this issue here open, but we will move this one to the newly created cwa-backlog repository. We will move it back once the colleagues can start working on concrete improvements.

Thank you very much for your understanding!

Mit freundlichen Grüßen/Best regards, SW Corona Warn-App Open Source Team

[kbobrowski]

@SebastianWolf-SAP thank you for the update, fully understand late answer!

The information which @kbobrowski received from Google is of course correct. The Google Framework - in theory - supports 'provideDiagnosisKeys' with more than 1 file per call. However, one requirement is that a batch number and the batch size that is adapted to the file amount sent to the method. Of course, the batch number would then be incremented with each file.

In the meantime I've implemented a small tool to find exposure notification instance (already initialized with app context) on Java heap and experimented with provideDiagnosisKeys (https://github.com/corona-warn-app/cwa-backlog/issues/2#issuecomment-647191033), so regarding limitations of that function there will be no more doubts from my side - also saw the limitations arising from batch_size and batch_num parameters that you describe.

Solving this issue without Apple/Google enhancements would require some changes not only in the apps, but also in the cwa-server backend.

Good news is that Google will implement passing multiple batches with only one quota taken (https://github.com/corona-warn-app/cwa-backlog/issues/2#issuecomment-647618727), which should allow for short update interval on Android without changes in the architecture (I'm not sure what is Apple's approach to this problem though - whether they plan to push similar update). Since it is supposed to happen soon there is probably little benefit right now in re-designing apps / backend to circumvent limitations of current framework - I guess we will come back to the discussion about it when Play Services update is pushed - it seems that then it will be just a small fix on the app side and no change on the server side

[kbobrowski]

@SebastianWolf-SAP couple more questions, would be grateful if you could also share your / team opinion on this:

• the iOS App queries the hour packages, but does not use them for the exposure detection

Is there any use for it right now, or is it just an artifact of previous approach? If this is the case then it is just generating unnecessary load on CDN.

• the iOS App schedules a job every 2 hours to check if a exposure detection is necessary

This is good that iOS is checking every 2 hours if exposure detection is necessary (it will check it also in the night, and user will wake up with notification about exposure). Android is also making progress by checking when user opens the app or refreshes Fragment (if there was no check during the day so far). The problem on Android seems to be that if someone is not actively opening the app then it may happen that notification will arrive late in the evening while all the necessary information is available already in the morning. Simple improvement could be to also have background job which checks every 2 hours if Diagnosis Keys have already been fetched that day.

changes are not that easy as we need consistency over both client platforms

Is this a formal requirement? Or is there some technical reason for it? I hope that Apple will also introduce multi-batching mentioned in my previous comment, but if this would not be the case I think notification time for Android users should not be limited just to maintain principle of consistency - faster notification for this group would benefit everyone around, especially considering that Android has 70-80% market share in Germany. Just my personal view on it - I guess we'll worry about it once Play Services update is pushed and Apple still does not have batching - but hopefully they will have it as well by this time :)

[SebastianWolf-SAP]

While I have no information about the rationale behind the internal differences - probably it‘s just because of different approaches at the beginning and the final goal to be consistent - I can comment on the last question, @kbobrowski: Yes, it’s a formal requirement.

Also if you ask me as a member of the Open Source team, I have to say that consistency makes sense. Everybody would have a hard time explaining the differences. An example: just think about the differences in age ratings between Google Play and Apple App Store just because the two parties apply different rating systems...

Mit freundlichen Grüßen/Best regards, SW Corona Warn-App Open Source Team

[kbobrowski]

@SebastianWolf-SAP I get that if users find out that Android updates exposure status every 2 hours and iPhone updates it every 24 hours it will result in some eyebrows raised and questions, but the answer seems to be quite simple - just official statement that it's a limitation of Apple framework and that Google allows to implement faster notifications, so for the benefit of everyone the decision has been made to fully utilize potential of each platform. I feel that users would understand (it's not really that difficult concept - for user safety it does not matter how fast user would get notification, but what matters is how fast contacts of this user would get notifications, and iPhone users obviously have contacts using Android phones).

But hopefully Apple will update its framework consistently to what Google plans to do - perhaps in the end there won't be a problem at all.

[kbobrowski]

@cfritzsche had interesting question:

Could the operator find out and disclose how many keys have been submitted yet below the threshold? Or how often a teleTAN was requested? Would be interesting if it’s more than 0.

It's quite concerning that there are no Diagnosis Keys published yet, just doing some simple analysis it seems that assuming 5M user base (to account for ramp-up to 10M), only 50 infections daily (to account for localized outbreaks where app might not be popular yet), and 50% probability of user sharing data with the system (I guess should be more in reality) - probability of at least one user sharing data after one week is 99%. Would be really interesting to know if any users submitted data so far.

>>> from decimal import *
>>> getcontext().prec = 32
>>> population = Decimal(83e6)
>>> user_base = Decimal(5e6)
>>> daily_infections = Decimal(50)
>>> p_one_user_diagnosed_daily = 1-(1-daily_infections/population)**user_base
>>> p_one_user_diagnosed_daily
Decimal('0.95080922140664735492147836858965')
>>> p_one_report_submitted_daily = p_one_user_diagnosed_daily * Decimal(0.5)
>>> p_one_report_submitted_weakly = 1-(1-p_one_report_submitted_daily)**7
>>> p_one_report_submitted_weakly
Decimal('0.98906619599593915805562451089320')

[christian-kirschnick]

@kbobrowski As you know, one user is not enough for a package to be published. I can confirm that there were already submissions and that we will see a package. Since the 1.0.8 patch is live now, we should see a file each day from now on.

[mh-]

@kbobrowski curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-23/hour/8 --output 2020-06-23-hour-8.zip

[nilsalex]

@mh- I'm confused -- I've repeatedly checked curl -X GET https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date for updates and up to now the result is empty. But it shouldn't be empty, as there is the bundle 2020-06-23/hour/8 available by now?

[mh-]

@nilsalex apparently the logic is different: To check if hourly packages are available, you need to query curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-23/hour for the specific day, independent of the result from .../date

[kbobrowski]

@christian-kirschnick right, I thought the old diagnosis keys would be pushed retrospectively with older date so they are immediately available to the users, but it's good that we'll see new packages every day now :)

@mh- nice, first package published!

@nilsalex yes date endpoint is still empty since it's only for completed daily packages, current package from /hour/8 will be bundled into /date/2020-06-23 only tomorrow, and only then it will be available for clients to do the matching with RPIs (https://github.com/corona-warn-app/cwa-backlog/issues/2#issue-643325481)

[cfritzsche]

@kbobrowski As you know, one user is not enough for a package to be published. I can confirm that there were already submissions and that we will see a package. Since the 1.0.8 patch is live now, we should see a file each day from now on.

Thanks @christian-kirschnick for the transparency.

[Spacefish]

Wow that implementation in the backend pushes notifications back even more.. Imagine somone entering a key today at 7:00, a device checks at 23:35 for new keys and get back nothing.. The next check will be scheduled ~24 hours later so for example 23:42 the next day..

So total delay is > 40 hours just due to implementation :-1: Even sending a paper letter across germany is faster than this...

why not provide a package for the current day in the backend which contains the keys already published that day? The device would have to pull packet today and today-1 the next day, but that would enable faster notifications...

I am under the impression that people did not really think this through before designing it like that. Furthermore the decision to keep iOS and Android consistent just for the sake of it (which it obviously isn´t currently, as iOS fetches hourly packages and don´t process them), is not a good decision IMHO.

[kbobrowski]

@Spacefish you are right that > 40 hours delay is possible, but it's due to implementation of the apps, not the backend. As far as I understand backend operates right now close to maximum possible efficiency - it publishes keys that are being shared by users on the same day, every hour. The problem is that apps wait for full daily bundle to be assembled - all hourly packages published by backend are bundled into one big package at the end of a day. If the phone checks after 23:00 for new daily bundle, and relevant Diagnosis Keys were already available yesterday before 01:00 then we have this 48 hour delay.

Just my personal opinion here: I think that trading fast notification time (increasing it from 3h to 48h) for possibility of estimating exposure score taking into account exposures from all downloaded packages from the last 14 days is not a good trade-off, I would prefer to be notified at shorter intervals, and if needed - on the expense of more false positives (as the "contact event" threshold would probably had to be a bit lower). But there is not much we can do about it right now.

Good news is that Google will push Play Services update soon, which would allow to combine these two approaches - have fast notification time (max delay about 3 hours) and also recalculate each time all packages from the last 14 days without the need to download all of them again (cache can be used, exactly like in current implementation).

[mh-]

curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-23/hour/13 --output 2020-06-23-hour-13.zip --> from 8:00 to 13:00, 4 unique users reported Diagnosis Keys - 5 days, 4 days, 4 days with old Android app (pre #679), 1 day --> 14 keys, multiplied by 10 by the padding logic. As expected, one cannot deduce which keys were reported by which of the (anyway anonymous) users.

[Spacefish]

Good news is that Google will push Play Services update soon, which would allow to combine these two approaches - have fast notification time (max delay about 3 hours) and also recalculate each time all packages from the last 14 days without the need to download all of them again (cache can be used, exactly like in current implementation).

With the current google implementation it is possible to archive 1,2 hours max. delay form reported key to notification, as you can submit 20 packages / day to the Exposure API. The rest is caused by a sub optimal implementation by SAP / Deutsche Telekom.

[kbobrowski]

@Spacefish you are right that with current Google framework it would be possible to achieve update every 2 hours - it would require dropping approach of feeding all packages for the last 14 days every time API is called, and instead feed just most recent packages (feeding any list of files is not possible, this list needs to be a coherent batch, see https://github.com/corona-warn-app/cwa-backlog/issues/2#issuecomment-647191033). I'm with you on this that it would probably be more pragmatic approach.

As for number of Diagnosis Keys expected, assuming 340 daily diagnosis (average for first half of June) and 10M users we should be seeing roughly between 30 and 50 users reporting a day, assuming everyone uploads TEKs.

[mh-]

% curl https://svc90.main.px.t-online.de/version/v1/diagnosis-keys/country/DE/date/2020-06-23/hour/17 --output 2020-06-23-hour-17.zip

% ./parse_keys.py -l -d 2020-06-23-hour-17.zip -u
(...)
Parsing the Diagnosis Key list, counting unique users...
Length: 190 keys (19 without padding)
4 user(s) found.
They submitted these numbers of keys:
1 user(s): 1 Diagnosis Key(s)
1 user(s): 3 Diagnosis Key(s)
1 user(s): 6 Diagnosis Key(s)
1 user(s): 9 Diagnosis Key(s)
Old Android app used by 2 user(s).
0 keys not parsed.

[cfritzsche]

@mh- Thanks for digging into this. I wish there was a chart that could be easily referenced showing the users per day uploading keys through CWA.

[MikeJayDee]

The user with 9 diagnosis keys is interesting. That person was either part of the app testers, or had another app installed (Immuni, SwissCovid, Apturi Covid, etc) as the CWA has not been available for more than 7 days.

In any case: Great that keys are being distributed, that the padding seems to be working and that delays have at least been reduced to at most a day and a bit. This should hopefully be visible soon in the iOS COVID-19 settings, where one should be able to see that a check has been made.

[daimpi]

@cfritzsche: @Spacefish might be working on setting something like this up: https://github.com/corona-warn-app/cwa-documentation/issues/258#issuecomment-646086866

[mh-]

The user with 9 diagnosis keys is interesting.

Well, how to assign key batches to users is ambiguous, and the script is rather simplistic - so your observation probably means that the script’s decision was wrong and keys were uploaded by one of the other users.

[geos-github]

How do you derive that someone has been using an old Android app? Does it use a slightly different data format?

[mh-]

@geos-github By this definition, "Old Android apps" had a significant problem in that they used the RKI's Transmission Risk Level profile in reversed order (see https://github.com/corona-warn-app/cwa-app-android/issues/678). I fixed that: https://github.com/corona-warn-app/cwa-app-android/pull/679, but some old apps seem to be still out there...

[kbobrowski]

Relevant paper about delays in contact tracing system: https://science.sciencemag.org/content/sci/368/6491/eabb6936.full.pdf , especially figure 3 on page 5. There is also interactive version ("interventions" tab): https://bdi-pathogens.shinyapps.io/covid-19-transmission-routes/

Plots below show growth rate assuming 24h, 48h and 72h delay from first symptoms to quarantine of the contacts, using default epidemiological paramteres. Assuming 24 hours between first symptoms and test results (optimistic, but should be possible with new rapid tests), this corresponds to no delay, 24h delay and 48h delay between infected person reporting and contact people receiving exposure notification - clearly shows deterioration of effectiveness of the system with each 24 hours passing.

[christianbrb]

Do you think the following issues in the ios repo are related to this problem?

• https://github.com/corona-warn-app/cwa-app-ios/issues/766
• https://github.com/corona-warn-app/cwa-app-ios/issues/759

[kbobrowski]

This pull request: https://github.com/corona-warn-app/cwa-app-android/pull/1088 enables significant reduction of exposure notification delay, as right now it's possible to call executeAPISubmission method up to 20 times a day, instead of only once as it used to be. Perhaps it would be good to call it 6 times per day (as this would be the limit for the exposure window mode - in case the change is going to happen soon), and include querying of hourly bundles.

This would reduce current worst-case delay (between one person submitting DKs and another person receiving exposure notification) of 48 hours to 5 hours (since hourly bundles are packaged every hour and would be checked every 4 hours). Average-case delay would be reduced from 24 hours to 2.5 hours.

@svengabr linking you here as well, not sure who is in charge of this issue right now

[svengabr]

Hey @kbobrowski

This topic was indeed was discussed during yesterday's developer calls. We all agreed that this needs a higher priority and we will start a new discussion with the stakeholders. Taking a look at the current increasing number of Covid-19 infections we need to lower the delay of exposure checks.

Unfortunately, there is nothing more that I can add to this discussion at the current point in time but I will keep this updated.

[axxel]

Disclaimer: This is going to be a rant but still meant in good spirit and with the intention to provide a view of how an interested outsider perceives the development of the whole project and this issue in particular.

I'm a software developer with a keen interest in how we as a society can manage this pandemic both from a scientific and a political point of view. Once it became clear (pretty much ever since the 'Oxford study') that the most crucial factor in breaking infection chains is quarantining people as early as possible, ideally 2 days before they show symptoms and about 3 days after they got infected, my hopes were high that a technical solution (likes this one) could make a real impact in obsoleting the rather 'dumb' brute force strategy of a general lock-down. Additionally, the insight that about about 50% of all infections a person will ever cause already happen in those 2 days, showed how important speed is.

Based on that information for me (as an outside laymen, of course) the two most important aspects of this whole endeavor were:

1. get as many people as possible to use the system and
2. warn them as early as possible.

I've been following this project from day 0 and there have been a number of moments where I thought something along the lines of "they can't be serous!?!". (To give just one example: the app tried a download once a day but if I'm offline at that particular moment, it'll wait another 24 hours?). In the context of this discussion here, there were a number of decisions made that obviously did not prioritize the speedy delivery of information (like let the limitations of the Apple API affect the usage of the Google API or prioritizing the possibility of modifying the risk model for past encounters for the price of adding a delay of about a day to the most basic and important functionality this whole thing was designed for in the first place).

So from my outside point of view, the progress that has been made in the last couple of months, especially with regards to this particular issue, is a huge disappointment (to put it politically correct ;)). Please: don't let this project dye. Because to me, that is exactly what it looks like what is effectively happening.

[daimpi]

Also potentially interesting in the context of this issue @EvgeniiSkrebtcov wrote:

Further more, we are preparing for the upcoming Exposure Notification API change (see description for getTemporaryExposureKeyHistory() method):

Planned for a future release but not currently activated in v1.5: The keys returned include the current day’s key, which is tagged with an expiration date (rollingPeriod). When the current key is released, the device stops using it to generate BLE beacons, and replaces it with a new key. Calling this causes a new key to be selected for the remainder of the current day. As a result, if this method is called multiple times, several keys with different rollingPeriod and identical rollingStartNumber can be released for those days it was called on.

I.e. it looks like there will even be a way to distribute "same-day DKs" in the future 🙂.

[kbobrowski]

@axxel right, it seems that risk calculation algorithm was designed just for providing the most accurate risk using the available information, not taking into account at all time-to-notification and API limitations.

What is a bit unfortunate here is that risk model is not something which is evolving over time to adjust to API limitations, improve notification time etc, it was just established once at the very beginning and then frozen. But Google / Apple API allows now for more frequent update even with the current risk model, hopefully the implementation will take advantage of it soon.

[mss1010]

To @axxel´s comment: I fully agree. In the last weeks, the progress of the project looked pretty much like polishing, updating texts and "minor" things like that, instead of focusing on the most important core goals and functionalities.

However I have the feeling, that the main blocker here is the RKI, which does not set the right priorities in my mind.

[kbobrowski]

@mss1010 right, RKI does not seem to be very agile in terms of actively improving risk model, which by the way seems to be based on flawed research paper - this error is known since 2 months but so far no known action on RKI side: https://github.com/corona-warn-app/cwa-documentation/issues/384.

But that's the water under the bridge at this point (in terms of adjusting risk model for faster notification, as notification time can be reduced by taking advantage of new EN quotas, still hope that RKI will take some position regarding mentioned transmission risk level error)

[christianbrb]

We all agreed that this needs a higher priority and we will start a new discussion with the stakeholders.

Stakeholder seem to be more important than users 😕

Who is owning the project and taking decisions?

[axxel]

It has been 3 weeks since @svengabr wrote

We all agreed that this needs a higher priority

And there has been zero progress on the subject. Not even an update on what might or might not have been discussed/decided in the background. The android app received a total of 5 commits during that time with 0 substantial changes. For me as a tax-payer who contributed to the 20 million EUR that went into the pockets of the people responsible for this, this is not enough. By any means. I don't expect the people reading this to be in a position to set priorities but they are in the position to talk to those who do. And they can at least report on the progress of the decision making process.

[svengabr]

This issue is not forgotten but this is a bigger task with many parties involved. We are still discussing this topic in our regular meetings and trying to progress further.

Some feedback:

It's probably the next significant feature that's going to be implemented but pending current negotiations. There's no ETA.

Another feedback

Decision might be dependent on data volume - architects

[axxel]

Here we are another two weeks later... and... still nothing. This (general) failure of making any significant progress with the app is starting to be noticed in the public, too: https://www.spiegel.de/netzwelt/web/corona-warn-app-das-teure-vergessene-mammutprojekt-a-39837ed8-adb0-4259-b98c-f5b417669efa.

[Ein-Tim]

@svengabr Could you please give us an update about this Issue? Is the Team still working on a more frequently download of new keys or is this not being worked on at the moment? Thank you!

[GisoSchroederSAP]

Just a quick update: Yes, we are aware on the risk that comes with the once-a-day sharing of keys. To make the CWA more robust, the QR code handling process more reliable for the users and to give better guidance to users regarding the risk level calculation and meaning, priorities changed in the latest weeks of development.

However, due to the rising number of Corona tests and the significat progress in international interoperability this topic now is on focus for the upcoming sprints in the management discussions and in the DEV planning. As the increased freqency will affect the data exchange bandwidth and the performance of the user's phone as well as the load on the server side, we still need to ensure reliability of the app with valid design and sufficient testing. Subsequently, there is no final release date or version yet, that we can communicate here.

[matzex]

Any update on this? With increasing infection numbers in germany all more early risk warnings are helpful to reduce new infections and stop spreading the virus.

[kyklos]

Is it really needed to delay in exposure notification due to artificial delayed fetching of Diagnosis Keys? Please change the default value from 24h to a lower than 4h to improve the quality of the contact tracing for every person.

I think the Telekom should be able to increase the bandwidth and allocate additional servers to their farm. The performance aspect for the phone should probably be neglected as the number of keys will not increase over the day.

[svengabr]

https://github.com/corona-warn-app/cwa-backlog/issues/2#issuecomment-711984017

Just a quick update: Yes, we are aware on the risk that comes with the once-a-day sharing of keys. To make the CWA more robust, the QR code handling process more reliable for the users and to give better guidance to users regarding the risk level calculation and meaning, priorities changed in the latest weeks of development.

However, due to the rising number of Corona tests and the significat progress in international interoperability this topic now is on focus for the upcoming sprints in the management discussions and in the DEV planning. As the increased freqency will affect the data exchange bandwidth and the performance of the user's phone as well as the load on the server side, we still need to ensure reliability of the app with valid design and sufficient testing. Subsequently, there is no final release date or version yet, that we can communicate here.

@matzex @kyklos as Giso already mentioned, we are currently working on this topic and its on our top priority list at the moment. This is not an easy change and it involves many parties because it will significantly increase the load on the environment.

[cfritzsche]

Thanks @svengabr . Really appreciate the progress. Unfortunately I still don't understand the infrastructure load argument. If 20 million people poll all records once a day or only 1/10th of the records 10 times per day should not make any difference in volume. What am I missing?

[GisoSchroederSAP]

@cfritzsche: The overall volume is the same, correct. However, the effective volume at a certain point of time will be lower on a user's device, depending on the frequency. If I poll twice a day, I get half the volume on my phone for each attempt. This has two effects:

• Less data volume on the local phone => faster processing, less performance load
• higher frequency of risk evaluation => faster warning in case of high risk

From my understanding two good reasons to increase the frequency.

On the other hand, yes, the server side definitely will have more load to handle: more requests, more activity to segment the data and more effort to avoid certain data or process issues (duplicates, collisions, locks, exclusions, congestions or similar).

[kbobrowski]

@GisoSchroederSAP I think server side is fully compatible with significantly increasing frequency in terms of data processing, as batching is already done every hour and server is making it available at /hour endpoint. Interestingly iOS app is already querying this endpoint, but not using this information for risk calculation, at least it used to be like this some time ago - see https://github.com/corona-warn-app/cwa-backlog/issues/2#issuecomment-647740143, @SebastianWolf-SAP wrote:

the iOS App queries the hour packages, but does not use them for the exposure detection

It seems to be just a question whether CDN can handle increased load from Android devices doing the same, which I assume would not be a problem.

Perhaps logic on devices would need to be adjusted not to feed full 14 day worth of DKs every couple of hours, but just do incremental check.

edit: not a swift developer, but it seems that iOS stopped fetching hourly endpoint since we last checked

[ndegendogo]

Perhaps logic on devices would need to be adjusted not to feed full 14 day worth of DKs every couple of hours, but just do incremental check.

@kbobrowski would the risk formula then yield the same result? I thought there was a summing step?