Make Low Risk Exposure display optional

my-tien commented 4 years ago

Avoid duplicates

[x] This issue has not already been raised before
[x] If you are proposing a new feature, please do so in CWA-Wishlist
I have carefully read https://github.com/corona-warn-app/cwa-backlog/issues/23
I am aware of the work in progress to add a better explanation to the app.
I am aware of the current stance of the RKI, cited in the above issue.

Current Implementation

The app shows low risk exposures and suggests the same behavior as in the case of no low risk exposures.

Suggested Enhancement

Hide low risk exposures per default as these can unnerve insecure people to a degree that leads them to uninstall the app. But add an option to turn it on.

Reasoning

I live with a woman in her 60s who received a low risk exposure notification. She was extremely upset by this and checked the app every day since then, hoping that the notification had vanished. She consulted with me, both of her sons and several friends about this and also read up on it herself.

Three of the people spoken to gave her extensive explanations about the meaning and implications of the notification.
I read to her the new explanation that is going to be added to the app.
In her internet research she read the CWA FAQ and other sources that explained the status accurately.
One person said that she didn’t know what the notification meant but that her husband had it too and ignored it.
One person mistook her inquiry as a search for the infected person and asserted that she was “clean”.
Two people told her that the app is stupid and that she should uninstall it.

After all this she was still insecure. She did not understand why a low risk notification is shown when no precautions need to be taken. Saying that its purpose is transparency, so that people can decide for themselves if they want to be extra careful, did not help. She simply didn’t know how she should decide to be extra careful or not. When she first brought up the idea of uninstalling the app because it made her anxious, I was able to talk her out of it. But several days later the notification was still there, creeping her out. She made the final decision to uninstall.

Her problem is clearly not a lack of information on the status, rather too much information combined with too much own responsibility and too little confidence on the topic.

Here’s another less anecdotal argument:

Although transparency is a good thing if I can make informed decisions based on it, it is not useful in and of itself. Since time, duration and location of the exposure are not shared due to data protection, there is no information to base a decision on. Hence, the transparency serves no purpose. If there are reasonable decisions one could make based on the notification alone, such as Refrain from meeting old people or Don’t go to work, they’d be the same for all low risk exposed people and could therefore be added to the behavior guidlines in the app. Apparently this isn’t the case, so I don’t see the purpose of displaying the exposure at all.

Expected Benefits

Keeping insecure people from uninstalling the app while satisfying the transparency wishes of other people at the same time.

Internal Tracking ID: EXPOSUREAPP-2834

ndegendogo commented 4 years ago

I like your suggestion to keep this feature as opt-in for "power users". I fully understand your reasoning that for many "normal" users this information has no or even a negative benefit. But for me the information of low-risk encounters helps to get more confidence in the working of the app (especially the closed-source parts).

MikeMcC399 commented 4 years ago

I was also presented with the information "Low Risk / 1 exposure with low risk" some days ago and even with my technical knowledge of how CWA works, I was still made nervous. If it had told me when this exposure occurred, I might have been able to think back and perhaps reconstruct where I was and what environment I had been in, maybe then thinking about avoiding the situation in future.

The information about low risk exposure is non-actionable. I was already abiding by AHA guidelines (Abstand halten, Hygiene beachten & Alltagsmaske tragen). I didn't change my habits based on the new status, so it did not help me.

ghost commented 4 years ago

Hello @my-tien,

thanks for sharing your idea. I have created a JIRA-Ticket and assigned it to the responsible person.

Thanks, LMM

Corona-Warn-App Open Source Team

my-tien commented 4 years ago

@GPclips @hermesmar Thank you. What is the process after moving the issue to JIRA? Will you keep us updated here?

heinezen commented 4 years ago

@my-tien Yes, by mirroring the issue to Jira it will get tracked by the developers and be discussed with the RKI. Any updates regarding your request will be made available here by the developers or the community team.

CH

Corona-Warn-App Open Source Team

daimpi commented 4 years ago

@my-tien regarding OP:

Hide low risk exposures per default as these can unnerve insecure people to a degree that leads them to uninstall the app. But add an option to turn it on.

I understand your request and the motivation behind it and in an ideal world I'd agree. Unfortunately there is at least one big problem I see with "hiding" those green encounters by default in the real world, let me try to explain: One important input which determines whether the risk of an encounter is low or high is the distance estimated by BLE attenuation. This process is very noisy at best: Fraunhofer testing had only about 50% recall rate, i.e. most of the other ~50% of actual infectious exposures would have been misclassified as green encounters.

The team from Ireland got even worse results in a real-world tram setup:

Conclusion:

We find that the Swiss and German detection rules trigger no exposure notifications on our data, while the Italian detection rule generates a true positive rate of 50% and a false positive rate of 50%. Our analysis indicates that the performance of such detection rules is similar to that of triggering notifications by randomly selecting from the participants in our experiments, regardless of proximity

i.e. basically all infectious exposures were misclassified as green encounters.

Or see this issue on "Impact of shadowing underestimated". I also recently played around with my Galaxy S8 as a sender (calibration-confidence: low) to check what kind of attenuation I'd see and I found that the offset seemed way too little for my device, as I almost all the time observed an attenuation >100dB with my Galaxy Tab S3 as a receiver (callibration-confidence: high) which yielded distances >8m even though the devices were almost next to each other and I took the respective offsets into account. Take this together with the fact that we've since learned that aerosolized spread of Sars-CoV-2 is a thing, and you can maybe understand why I'm getting very nervous when I hear suggestions of hiding green encounters by default.

As you're familiar with https://github.com/corona-warn-app/cwa-backlog/issues/23 you probably know that there was a suggestion to introduce a yellow risk status for such cases, potentially in combination with an "expert mode" which would show additional details on such encounters, and allow you to configure the threshold for notifications. To quote @kbobrowski:

Interesting, risk calculation in general feels a bit over-optimized and over-confident (e.g. registering contact but still displaying "low risk" with no notification if risk calculation did not cross the threshold). Perhaps it would work ok if we did not have such significant uncertainties in the system already (very short scanning windows + variability of BLE signal strength). What makes it further worse is that mathematics in "Epidemiological Motivation of the Transmission Risk Level" were probably based on a flawed research paper (related issue: corona-warn-app/cwa-documentation#384).

Maybe a solution which could be considered is to introduce third interim state "possibly increased risk" with yellow card and notification whenever at least 1 key has been matched

In a scenario with a yellow risk status and an expert mode + maybe additionally somewhat wider attenuation bucket sizes I'd feel much more comfy even if the default setting would be to hide yellow encounters.

Unfortunately this idea has since been rejected by the RKI 🙁.

This means we're left with all the problems mentioned above and under such circumstances I'm personally feeling much more comfortable with the idea that some users will get irritated by "non-actionable" information than I am with accepting that the significant share of false-negative encounters will not even provide a slight nudge (e.g. to take hygiene measures especially seriously) for users who left their settings on "default". I'd be fine with the option to hide green encounters in CWA if it's not the default setting though 🙂.

Ein-Tim commented 4 years ago

I personally would even go further and say: With all these Issues reported, these studys and researches (i.e. the Frauenhofer mentioned from @daimpi above) done, it makes no sense to have two Risk "classes".

If there would be one risk class, with more Information, like Date and Time (maybe even the Location) everybody would know where he/she has been and what he/she has done and everyone could make his own Risk assesment....

I know that it is nearly impossible (from technical side [ENF don't expose exact Time or even location] or from data protection) that this will be implemented like ddescribed above, but I still think that the Idea of having the App do a Risk Assecment for the User hast to be overthinked... (A very stupid question: Are all Apps using the ENF have 2 Risk States, or is the German one special here?)

ndegendogo commented 4 years ago

and everyone could make his own Risk assesment

How can an average, nontechnical user (like my mother ... ) make his/her own risk assessment if the app presents a lot of numbers - and that's it?

daimpi commented 4 years ago

@Ein-Tim

it makes no sense to have two Risk "classes".

While I do agree, that the distinction is quite noisy and the parameters governing this separation are currently not set optimally, I wouldn't go that far. Having some kind of default actionable distinction/information provided to you by the app is probably still an important function for the vast majority of users (like @ndegendogo's mother 😉).

And I think even a binary distinction could in principle be made to work quite well, with some parameter tweaking + some additional info in case of any encounter (at least as an option).

And while I personally would be happy if ENF exposed exact timestamps for encounters, I can also understand why Google/Apple don't want to do this out of fear of privacy violations and therefore only expose timestamps coarsened to daily intervals. Back in August we had some discussion on exactly this topic on Gitter.

Are all Apps using the ENF have 2 Risk States, or is the German one special here?

I don't really know that much about other apps, but SwissCovid has also a binary distinction afaik. And my guess would be that this is probably the case for most ENF apps.

Ein-Tim commented 4 years ago

@ndegendogo For sure the App would have to show some additional Information which are helping the User to do his own Risk Assesment better. But yeah I know what you mean (thinking about my mother 😅), but the question is:

What is more reliable? The User oder the App done Risk Assesment? For me personally this Question is at the moment answered with: Me myself.

(But reading @daimpi's comment again, I think the best would be an Expert Mode and a Normal Mode)

@daimpi

While I do agree, that the distinction is quite noisy and the parameters governing this separation are currently not set optimally, I wouldn't go that far.

On the second thought, you (and @ndegendogo) are both absolutely right, the most users will need a Assesment done by the App...

And I think even a binary distinction could in principle be made to work quite well, with some parameter tweaking + some additional info in case of any encounter (at least as an option).

Yeah, I also think so, but looking at f.e.: this Twitter Thread reporting that 2 phones from the same person show different Risk "Classes" because one of them was in a bag, this leaves a bad taste and also makes me uncomfortable thinking about my latest and first green Risk Encounter, since my phone is nearly always in my bag...

Back in August we had some discussion on exactly this topic on Gitter.

Thanks for the hint 👍

my-tien commented 4 years ago

Thanks for all your replies.

We can talk about the shortcomings of the Bluetooth service all day. The way I see it though, the most relevant fact is that the RKI has decided to not distinguish between low risk exposures and no exposures. One could open a separate issue to request that (again), but my suggestion works on the premise that no distinction is being made.

I am well aware of the Ireland study. But what percentage of the population do you assume to be knowledgeable enough about these limitations to make an informed decision? The problem doesn’t go away simply by showing some cryptic low exposure notification.

If the expert user feels confident enough to react to the low risk exposure notification, they should also be knowledgable enough to turn it on. But the default user should go with the default guidlines by the RKI and the app should reflect said default.

The number one priority for design decisions atm should be to increase the user base or at least not to shrink it.

daimpi commented 4 years ago

@my-tien thanks for your reply 🙂

The way I see it though, the most relevant fact is that the RKI has decided to not distinguish between low risk exposures and no exposures.

That's not entirely true though, is it? We can currently distinguish three different states in CWA:

low risk/green, no encounter
low risk/green, some encounters
increased risk/red, some encounters

And (2) now even has it's own dedicated explanation text which differs from (1) since CWA 1.3.1.

One could open a separate issue to request that (again) […]

As I said above: imho even a binary color coding could work quite well with some adjustments, I'm not insisting on three colors:

And I think even a binary distinction could in principle be made to work quite well, with some parameter tweaking + some additional info in case of any encounter (at least as an option).

Regarding

I am well aware of the Ireland study. But what percentage of the population do you assume to be knowledgeable enough about these limitations to make an informed decision? The problem doesn’t go away simply by showing some cryptic low exposure notification.

I'm all for showing more information to the user (cf. https://github.com/corona-warn-app/cwa-wishlist/issues/178, https://github.com/corona-warn-app/cwa-wishlist/issues/100) instead of "just some cryptic encounter", and I don't expect the average user to really understand all the limitations that come with any (red or green) encounter. But even in it's current form seeing a green encounter can provide a little nudge to the user, reminding them that the app is working and at the same time calling attention to the hygiene rules which the user should follow especially thoroughly in this case.

The number one priority for design decisions atm should be to increase the user base or at least not to shrink it.

In general I agree o/c that higher uptake is an important goal. But it's "just" an instrumental goal in service of the terminal goal: to stop the virus from spreading by breaking chains of infection. And given the current shortcomings of CWA I've described above (mainly the low recall- / high false-negative-rate) I'm afraid that exactly this terminal goal of breaking chains of infection would be compromised if we were to hide green encounters by default. As I said above: if those shortcomings were addressed, I'd feel much more comfy with the idea of hiding green encounters by default. Additionally: even if we're just talking about the instrumental goal of trying to increase adoption, I'm not really sure that hiding green encounters is necessarily beneficial there: Yes some ppl will get so irritated that they uninstall the app, but on the other hand there will be ppl for whom this is a nice confirmation that the app is actually doing it's job and who post about this on social media, which increases brand awareness for CWA and in turn might lead to higher adoption.

kbobrowski commented 4 years ago

If i remember correctly RKI wanted to hide green encounters, but in case there is a green encounter ENF will include this information in weekly summary and in exposure logs, which would be confusing for the user - why there is information about encounter in ENF but not in CWA. I think this is the main motivation of showing green encounters in CWA at all

ndegendogo commented 4 years ago

ENF will include this information in weekly summary and in exposure logs, which would be confusing for the user

well - the new ENF implementation since iOS 13.7 does no longer show these matches. I hope they did not intentional hide this for a similar reason, but that it's 'only' a bug that they will fix. And, yes, I agree, this kind of inconsistency is very confusing and irritating.

MikeMcC399 commented 4 years ago

On Android CWA 1.3.1 the string risk_details_additional_info_text has been changed to:

"You encountered a person who was later diagnosed with COVID-19. Nevertheless, based on your exposure logging data, your risk of infection is low. The risk is low if your encounter was brief or occurred at a distance. You do not need to worry and there is no specific need for action. We recommend that you adhere to the prevailing rules regarding distancing and hygiene."

Given that it was decided to continue to show this green low-risk encounter, the text does at least now say, "You do not need to worry". I can understand the logic mentioned by @kbobrowski in https://github.com/corona-warn-app/cwa-wishlist/issues/181#issuecomment-699584741 that if a match is showing up in the ENF exposure logs it somehow needs to be dealt with visibly by CWA. Requesting somebody not to worry may however not stop somebody worrying!

ndegendogo commented 4 years ago

your risk of infection is low. The risk is low if your encounter was brief or occurred at a distance. You do not need to worry ...

@MikeMcC399 so you think this is not clear enough? What are you missing?

MikeMcC399 commented 4 years ago

@ndegendogo The message is clear and I wasn't requesting any change. I was just following up with the information about what has changed in the warning string.

Luckily I have not had any more encounters and I'm happily seeing "No exposure up to now" on my phone at the moment. :smiley:

ndegendogo commented 4 years ago

So, today I got my "red" notification. It shows 6 encounters, and latest is 3 days ago. Now I am very happy that I had seen the "green" contacts before. Yesterday 5 green; today 6 and it shows red. And I can verify in the ENF logs: 1 match in the new file (so this is the red), and still 5 in the older files. Because: 1 red and 5 green sounds less intimidating than 6 red.

And: yes, tomorrow I will follow the recommendations and get advice from the health authorities (Gesundheitsamt) how to proceed.

MikeMcC399 commented 4 years ago

@ndegendogo Good luck with the Gesundheitsamt! 🤞 🍀 RKI publishes a document CORONA Warn-App - Zur Information für die Mitarbeiter/innen der Gesundheitsämter in case you didn't know about it.

ndegendogo commented 4 years ago

@MikeMcC399 Thanks!

Actually, I tried the 116117 today, but the number was completely congested ... after ~1/2 hour I got the line, but at that moment my connection broke down ... I'll wait till Monday now. And, yes, in the meantime I am self-isolating.

ma137 commented 4 years ago

Dear @GPclips @heinezen @JoachimFritsch ,

what is the current status of this enhancement request?

I think @my-tien made a very good point when she/he says that the information about the low-risk encounters make some users nervous without any need and that this behavior of the app is at risk of shrinking the user base. I know two persons which consider uninstalling the app because of the frequent low-risk encounters. While such a reaction might not be rational, it is very human. Therefore, the implications of these "notifications" about low-risk encounters should be considered. Hiding the low-risk encounters per default is in my opinion a good solution to this problem.

Thank you for your great work on the app! ma137

my-tien commented 4 years ago

So, today I got my "red" notification. It shows 6 encounters, and latest is 3 days ago. Now I am very happy that I had seen the "green" contacts before. Yesterday 5 green; today 6 and it shows red.

That's an argument for improving the display of high risk contacts, not for displaying the low risk ones.

Actually I think your problem reveals a major flaw of the app, and if the issue doesn't exist yet you should open one.

Ein-Tim commented 4 years ago

@my-tien

Actually I think your problem reveals a major flaw of the app, and if the issue doesn't exist yet you should open one.

There is an Issue for this: https://github.com/corona-warn-app/cwa-documentation/issues/422

ndegendogo commented 4 years ago

@Ein-Tim actually, corona-warn-app/cwa-documentation#422 is a slightly different topic. And, yes, I agree, both issues are confusing and lead to wrong conclusions.

Example 1: lets assume, I have a red 3 days ago, and a green 5 days ago. Then it will show red (correct), and tell me: "2 risk encounters" (although only one was high risk), "last was 3 days ago" (which is again correct).

Example 2: now lets assume, I have a red 5 days ago, and a green 3 days ago. It will show red (again, correct), and tell me: "2 risk encounters" (wrong as above), "last was 3 days ago" (although the high-risk was 5 days).

Example 1 is focus of this ticket. Example 2 is focus of documentatio#422; and has even more serious impact, because: if I take the test too early it could be false-negative; so I could decide to wait another 2 days before I get tested; but then I loose these 2 days before I get my result and upload my own keys and warn my contacts.

daimpi commented 4 years ago

Related requests which would alleviate the situation:

my-tien commented 4 years ago

Example 1: lets assume, I have a red 3 days ago, and a green 5 days ago. Then it will show red (correct), and tell me: "2 risk encounters" (although only one was high risk), "last was 3 days ago" (which is again correct). […] Example 1 is focus of this ticket.

Actually, no. I wasn’t even aware of the ambiguity problem you talk about. Although it’s an important problem, I think you should rather discuss it in the linked issue, or – if you feel it doesn’t belong there – in a separate issue.

This ticket focuses on the usefulness of the low risk encounter for the broader public in general. Let’s keep the discussion here on topic.

ohobby commented 4 years ago

From my own experience, I can report that reporting that you have had one or more low risk encounters, but you don't know when it happened, causes many people to panic and/or brood a lot about when and where it happened. My partner and some friends are such candidates. For this reason it would be very important that the date of the encounter is mentioned very quickly, even in the case of low risk encounters. This would help some users not to panic or brood. And prevent many requests to the health department and/or doctor.

daimpi commented 4 years ago

@ohobby

For this reason it would be very important that the date of the encounter is mentioned very quickly, even in the case of low risk encounters.

See this issue. Feel free to upvote 🙂.

Ein-Tim commented 3 years ago

Since CWA 1.9.1 and the switch to version 2 of the ENF/ENS very low risk encounters are now dropped by the App and not shown to the user. Thus, the number of low risk encounters shown to the user dropped heavily.

my-tien commented 3 years ago

Hm, what is the difference between low and very low lisk encounters?

Ein-Tim commented 3 years ago

@my-tien

Those very low risk encounters were even under the CWA defined limit for low risk encounters, so they were never supposed to be shown to the user. But since it was not possible to hide these encounters in version 1 of ENF, all of those were shown to the user. This has now changed.

mfouesneau commented 3 years ago

I've never seen a "very low risk" message before.

ndegendogo commented 3 years ago

I've never seen a "very low risk" message before.

Yes. They were all just green. You could not tell whether such a green encounter was a "1-second contact" at large distance, or "just below the threshold to red".

MikeMcC399 commented 3 years ago

@my-tien If you want to understand more, then I recommend the blog entry https://www.coronawarn.app/en/blog/2020-12-30-cwa-behind-the-scenes/

which contains links to:

a lecture with
slides particularly starting at slide 13.

There is also another blog post Risk calculation under Exposure Notification Framework Version 2 with helpful explanations.

You can read the second blog post together with lecture slide 18. The slide shows an example of an encounter which is dropped because it does not meet the criteria of attenuation less than 73dB (assumed equivalent to closer than 8m distance) for more than 10 minutes of a 30 minute window, combined with a transmission risk level (TRL) of 3 or more. If the criteria are not met then this is labelled a "non-risk encounter" according to the explanation in the blog.

According to https://github.com/corona-warn-app/cwa-documentation/blob/master/cwa-risk-assessment.md "each dB value is associated with a particular distance".

Attenuation	Distance
55 dB	1.5m
63 dB	3m
73 dB	8m

corona-warn-app / cwa-wishlist