corona-warn-app / cwa-documentation

Project overview, general documentation, and white papers. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
3.28k stars 344 forks source link

[Business Rules Validation]Certificate validated as eligible for travelling despite Country of Arrival has no business rules in PROD #678

Closed vaubaehn closed 3 years ago

vaubaehn commented 3 years ago

Avoid duplicates

Technical details

Describe the bug

When you check the validity of your EU DCC against business rules of another country, and that country didn't provide any business rules via the gateway yet, nevertheless title and headline of the screen states "Zertifikat gültig". This should not be the case, as "we" (as in CWA and the user) do not know whether the checked DCC is egliable for travelling in that country without looking up on https://reopen.europa.eu/de . Example for Belgium, which still not provided any business rule:

This is even worse for Austria, which has some specific rules for their vaccination certificate: 1/2 is valid, but only when first vaccination is more than 21 days ago, but less than 90 days ago. As Austria did not provide business rules for PROD yet, just by reading title and headline of the screen, people with a first vaccination taken less than 22 days or longer than 90 days ago might think "everything is fine" while it is actually not. Same is true for any test, recovery and 1/2 vaccination for other countries without business rules in the gateway.

Imho, the following explaining text "Für diesen Zertifikattyp sind derzeit keine Einreiseregeln für das gewählte Land vorhanden. Bitte beachten Sie die folgenden Hinweise." can also be misinterpreted like "you may enter the country with any DCC, as the country did not set up any rule".

Steps to reproduce the issue

  1. In CWA, select a certificate that you know it's not valid in a CoA, and CoA has no active business rules in PROD yet.
  2. Click on "Gültigkeit überprüfen"
  3. Select CoA without active business rules.
  4. See error/misleading information

Expected behaviour

CWA must not state that DCC for the CoA is valid, even no rules have been checked.

Possible Fix

Two options are coming to my mind:

a) Imho the best one: Do not validate DCC for countries that have no business rules in PROD. For this, the selector that selects the country for validation must only provide countries that have business rules in PROD (the list would need to be filtered accordingly before the screen is displayed).

- or -

b) Change the headline and title of the screen: "Prüfung nicht möglich"/"Das gewählte Land hat keine Regeln für eine Prüfung zur Verfügung gestellt" in a non-green color scheme (could be gray or blue for example).

Additional context

Above I gave an example of Austria. For Austria it's actually more complicated, as they now have some countries of departure (high incidence/high risk areas) that are excluded from the (business) rules. AFAIK there is yet not way to validate such rules via ehn-business-rules/CertLogic. The "Hinweise" below the validation result should reflect these kind of exceptions more prominent - for now there is "just" a small bullet point "Es können in einzelnen Regionen weitere Regeln oder Einschränkungen gelten". The conclusion here is, that users should always additionally check on https://reopen.europa.eu/de for other restrictions, and that should be communicated very clearly in the validation screen.


Internal Tracking ID: EXPOSUREAPP-8826 Internal Tracking ID: EXPOSUREAPP-8785 related Internal Tracking ID: EXPOSUREAPP-9018 Internal Tracking ID: EXPOSUREAPP-9019

dsarkar commented 3 years ago

@vaubaehn Thanks, Internal Tracking ID: EXPOSUREAPP-8826

Jo-Achim commented 3 years ago

@vaubaehn I think the proposed solution b) would be the better one. The countries without "business rules in PROD" may additional be grayed out for identification.

The complete hiding of countries without "business rules in PROD" (a)) would only lead to the unnecessary question whether the country list is complete or why countries are missing.

dsarkar commented 3 years ago

@vaubaehn @Jo-Achim FYI This issue is with the stakeholders for alignment.

DooMMasteR commented 3 years ago

Is there any progress? This issue is more on the "not cool" side of issue, especially since the text might be misinterpreted as if there were no rules in the selected countries, when instead the app could just not validate it at all.

GisoSchroederSAP commented 3 years ago

Just a quick summary:

  1. Currently, the app decides the validity as a result of verification process based on the business rules of the respective country. As long as there are no restrictions defined, the certificate seems to be valid (like in many firewall configurations: If something is not restricted by rule, then it is allowed, right?).
  2. However, we already document the hint, the user is expected to double-check the current rules on the ReOpen.EU website shortly before traveling - just like the hint that is already mentioned on the paper certificate. So, the user cannot transfer responsibility to the app.
  3. Also, I'd like to point out: The Corona Warn App is not the official verification app of Germany. So, verification by country-specific rules is a convenient feature and does not replace the checking by any other national Check App.
  4. Still, we are going to introduce a third status of validity checking in the CWA (contrary to "firewall comparison") with a clear title "Cannot be verified" and a question mark as symbol - to make it even more clear.
  5. What will not happen, though: That we will implement a "free text reading and processing" of the published national statements on the ReOpen.EU website or on any other place. I hope, this is a commonly-accepted decision.

From Community Mgmt we emphasized the expectation this enhancement in 4. becomes available soon. Keep the fingers crossed...

Ein-Tim commented 3 years ago

Still, we are going to introduce a third status of validity checking in the CWA (contrary to "firewall comparison") with a clear title "Cannot be verified" and a question mark as symbol - to make it even more clear.

Seems like this will be implemented in 2.8: https://github.com/corona-warn-app/cwa-app-ios/pull/3371

DooMMasteR commented 3 years ago

Soo much better :+1:

vaubaehn commented 3 years ago

Related Android PR: https://github.com/corona-warn-app/cwa-app-android/pull/3932

vaubaehn commented 3 years ago

Hi @GisoSchroederSAP , welcome back! I hope you had a nice and relaxing vacation!

Personally, I'm copying all your points 1 to 3 above. I guess we are agreeing, that for you and me and all the other tech savvy and well-educated users here in the CWA community handling this special feature does not leave many questions open, on how to interpret the currently shown result and how to proceed.

But I was opening this issue for a big proportion of other users, who may have more difficulties to understand what these results mean with regard to checks for countries without validatable business rules. From my observations and experiences I concluded, that it would be necessary to change the functionality in this specific point, to be as much 'crystal clear' as possible to avoid any misunderstandings in the whole user community for this quite important feature. And I think my findings are backed by some (duplicate) issues in our repos here, and also by some user reports/reviews in (Android) store.

Anyway, it's good that it get's implemented. 👍

Edit: Just one short side note:

So, the user cannot transfer responsibility to the app.

This is right. But it is also about, what a user can expect from the app. And in this special case, to have realistic expectations of what the app can do for you here, it would also need some basic understanding and knowledge about the 'system', the backend and the workflow, that this feature was built on top. And that's not really easy for everybody to understand. So, again, good that it's going to be enhanced.

Jo-Achim commented 3 years ago

I would like to underline @vaubaehn's point of view above. Against the background of the many open CWA expansion requests, the CWA must remain clear, understandable and easy to use - even for people who are not very technically savvy, older people, etc. In this way: keep it simple.

As a second point, I would like to add, purely as a prophylactic, the wish not to introduce "non-binding parallel instances" into the CWA via functions / features. For example, according to the motto: You can check something here (without obligation), but you only get it binding if you also look 'there'. Then the sole reference to look 'there' would, in my opinion, make more sense.

GisoSchroederSAP commented 3 years ago

As much as I agree to the expectations above, I still need to make this clear: In order to stay simple, one major stakeholder decision was, that the checking/verification officially is not a major task for the CWA, same applies for the certificate "translation". Many of the discussions theses days, however, only focus on misunderstanding, misinterpretation of certificate details, including the verification and validity checking/result. I think, meanwhile we have a gap between user expectations and scope/purpose of the app.

So, if you expect, if the majority of the users expect that all this should become a main function within the app, then we need to forward this expectation to the stakeholders - we may start a discussion, whether or not CovPass and CovPass check are required in the future, if the CWA incorporates their functions. Correct?

Alternatively, we also could think about the easy way and remove those functions from the app - if they generate more confusion than benefit for our users.

In the end, we decided, the certificate handling and additional features will make the CWA more valuable, even besides other apps with partly overlapping functions. Therefore, our focus now is to provide UI, communication, and documentation in a way, that can be consumed by the users.

vaubaehn commented 3 years ago

@GisoSchroederSAP Again I am agreeing to your points above, but I'd like to invite you (and everyone else) to look at some aspects in a different, more benevolent light.

In order to stay simple, one major stakeholder decision was, that the checking/verification officially is not a major task for the CWA

Imho, this is absolutely understandable and a good decision. But I think, to additionally introduce the "wallet app feature" was one of the most excellent decisions in the whole development process, for several reasons: this "goodie" not only gained a respectable amount of new users for the CWA, but seemingly re-activated an even bigger amount of users, whose CWA installations were sleeping for months, because they could not find much advantage by the major feature - the contact tracing. This is one impression I got from frequently reading reviews in Android app store. While some months ago the majority of reviews had not been so cheerful, most of the reviews from the past weeks were very positive! And this supports the major feature/the main goal of the app: to get as many people as possible excited about the app, to use the contact tracing and hence to break infection chains.

Many of the discussions theses days, however, only focus on misunderstanding, misinterpretation of certificate details, including the verification and validity checking/result.

This is true, currently. But when I am reflecting the development process of DCCs and Business Rules as a result of political decisions, this was most likely unavoidable: there was not much time for development in general, many changes needed to be introduced into a running process, while politics urged to release first versions of DCCs and Business Rules as "ready to the market" prior to extensive field testing. As Gemany decided to start issuing DCCs before the official release at July 1st, from my perspective we were experiencing a big beta test for the whole EU here. And, frankly speaking, I think SAP/TSI made an extraordinary job with a fast implementation of these features. But to be able to do so, not every issue could have been anticipated, not every sentence could be phrased perfectly. This is why I'm seeing these discussions, misunderstandings, misinterpretations as a temporary problem. In my eyes, solving them could be seen as "fine tuning". And we're here to help, if possible.

So, if you expect, if the majority of the users expect that all this should become a main function within the app, then we need to forward this expectation to the stakeholders - we may start a discussion, whether or not CovPass and CovPass check are required in the future, if the CWA incorporates their functions. Correct?

I do not expect so. But honestly, I think some parts of the users experience the additional features as the main features for themselves. And I think, this is legitimate - because it's also supporting the main goal: to have as many people as possible for contact tracing also, to break infection chains.

I think, meanwhile we have a gap between user expectations and scope/purpose of the app.

I see most of the people very happy in their reports, generally without demand for any new 'big' features. For me it looks like, currently user expectations and scope/purpose of the app are aligned in the best way possible.

Alternatively, we also could think about the easy way and remove those functions from the app - if they generate more confusion than benefit for our users.

I don't know how you felt when you wrote these lines - but removing these features from the app won't help anyone. We will overcome remaining confusion soon, just by enhancing some sentences and UI with small effort.

In the end, we decided, the certificate handling and additional features will make the CWA more valuable, even besides other apps with partly overlapping functions. Therefore, our focus now is to provide UI, communication, and documentation in a way, that can be consumed by the users.

That said: let's finetune our app!

I'm very, very, very convinced that Corona-Warn-App saved and still saves lives. The additional features supported it.

GisoSchroederSAP commented 3 years ago

Thanks for the flowers to the folks implementing and promoting the CWA, @vaubaehn. Obviously, I totally support the idea of finetuning the app whenever possible - in function, in UI, in documentation, and you valuable input here is much appreciated!

I just want to avoid that we all solely discussing rather the candy on the cake than the cake. So, I hope, it's "crystal clear":

I hope, we all can agree on this. Thank you for your valuable support.

MikeMcC399 commented 3 years ago

@vaubaehn

The blog announcing the release of CWA 2.8 https://www.coronawarn.app/en/blog/2021-08-25-cwa-version-2-8/ explicitly mentions the change which resolves this issue:

"In addition, the project team has adapted the EU certificate check. If a country has not provided any entry rules that the Corona-Warn-App can use for checking, the certificate check will state that the certificate cannot be validated. The countries that are listed under the certificate check in the Corona-Warn-App generally recognize the certificates. However, depending on the country and the applicable rules, the relevant certificate (on its own) may not authorize entry.

Previously, the app indicated in this case that the certificate is valid in the selected country. In the text below, however, it was pointed out that there are currently no entry rules available for the selected country. With version 2.8, users can clearly see that the certificate's validity could not be checked because the relevant country has not defined any entry rules."

I have checked that this is fixed in the released version CWA Android 2.8.0 (see https://github.com/corona-warn-app/cwa-app-android/issues/3922#issuecomment-906247239).

Thanks for highlighting this issue!

Ein-Tim commented 3 years ago

Confirming fixed under iOS with version 2.8.0, released yesterday, 25.08.2021.

vaubaehn commented 3 years ago

I'm still on CWA 2.6.1 🤷‍♂️ , but as @MikeMcC399 stated here: https://github.com/corona-warn-app/cwa-app-android/issues/3922#issuecomment-906247239 this issue is solved also for Android. So I guess, we can close.

Thanks to everyone involved to achieve a better UX! ❤️

vaubaehn commented 3 years ago

There are now dozens of reports like this in Google Play Store:

Seit dem letzten Update (Version 2.8) funktioniert die Gültigkeitsüberprüfung für andere Länder nicht mehr. Angeblich liegen keine Einreiseregelungen für das gewählte Land (hier Frankreich) vor. Die haben sich aber meines Wissens nach nicht geändert. Bitte Fehler dringend beheben!

😱 😢 🤷‍♂️

Maybe that wouldn't have happened with

a) Imho the best one: Do not validate DCC for countries that have no business rules in PROD. For this, the selector that selects the country for validation must only provide countries that have business rules in PROD (the list would need to be filtered accordingly before the screen is displayed).

but also that solution would have caused confusion due to a changed behavior.

Anyway, it's out now. Although I'm feeling a bit guilty to have brought up this issue and proposed to 'fix' it, I think all the reactions just show, that these people have clearly misunderstood the "certificate valid" state for checks without business rules, as they now consider the certificate to be invalid, which it is not.

@dsarkar @heinezen What could be the best way out? For a fast reaction I guess it would be good to bring up a clearifying FAQ entry, that can also be used by the store support teams to point to in their answers. Would you agree to that? If yes, I'm a bit scarce in time currently, and would be happy, if someone could jump in for it...

Second question would be, if in-app heading/texts could be enhancend to better clarify what is this about. Heading (in German) could be replaced with "Einreiseregeln nicht prüfbar". Text could maybe explain a bit more in one or two sentences and additionally point to the corresponding FAQ? If yes, this would probably best to handle in a follow-up issue?

Edit: For me the FAQ https://www.coronawarn.app/de/faq/#cert_eu_travel is already quite clear. Maybe it can be enhanced a little (e.g., explaining: yes, every country has "Einreiseregeln" but until today only few provided them to a special server gateway that allows CWA to use them for validation) and information that the behaviour changed with 2.8., how results are presented when no business rules are stored in the IOP gateway by the participating countries.

vaubaehn commented 3 years ago

Re-opened to clarify whether and how to proceed with user confusion.

Ein-Tim commented 3 years ago

@vaubaehn

I also talked to very confused users on Twitter. I will start preparing a PR updating https://www.coronawarn.app/de/faq/#cert_eu_travel now.

vaubaehn commented 3 years ago

@Ein-Tim Thank you! In general, the blog entry, that @MikeMcC399 pointed to (https://www.coronawarn.app/en/blog/2021-08-25-cwa-version-2-8/) is also not that bad. I think what is causing most confusion is, that all information before did not make clear, that next to the "Einreiseregeln" that have been published by member countries in "media" (internet and alike) are not sufficient for CWA. The countries need to translate these rules technically and provide them via the IOP gateway, so that CWA can download them and use them for validation. And that the problem was before v2.8.0 that CWA did not make this clear and it could have been misunderstood due to the form of displays of results. that the cert may be accepted (e.g., "Zertifikat gültig" for incomplete vaccination) just because of the form of display when no rules are present, while it would not be accepted actually...

I know you understand me at least...

Ein-Tim commented 3 years ago

@vaubaehn

What about this?

Bildschirmfoto 2021-08-29 um 01 16 46

vaubaehn commented 3 years ago

For me this is all clear (as it was also before). But I don't know, if some users will still struggle...

Ok, I decided to continue procrastinating from my work that I would need to do, and to continue on this. In general I like, what you wrote, I will try to add/rephrase some parts for even more clarity:

Update With version 2.8 the behaviour of the certificate validity check has changed. Entry rules for travellers have been set up by all EU member countries, but until today only a few countries provided technically translated entry rules which Corona-Warn-App can make use of. Before version 2.8, when there were no rules available to be downloaded for a certain country, after a check Corona-Warn-App displayed "Certificate Valid" and explained in the following text, that for this country no rules were provided/could have been checked. "Certificate Valid" referred to "Technical Validity" in this case [insert FAQ LINK here]. However, we understood that this result may be misinterpreted, for example when an incomplete vaccination was checked, the wrong conclusion could have been drawn, that this certificate warrants entry without restrictions. We clarified [sic. I was spelling it wrong before] this since version 2.8: In case a member country did not provide technically translated entry rules for Corona-Warn-App, the headline now states "Certificate Cannot Be Validated", whereas the following texts stayed the same. This does not mean that your certificate is not valid or that you are not allowed to enter the country you checked your certificate against. In this case, please verify your status with information that you find in https://reopen.europa.eu/. Further [SPELLING] information on this change can be found in the second paragraph [SPELLING] of this blogpost: [LINK]

It's a bit longer, but maybe hopefully now clear for everyone?

Thanks for your support! Need to return to my work now...

DooMMasteR commented 3 years ago

Then maybe just show those countries in the list, which have published a rule set and add re reopen.eu info below the initial check dialogue?

On Sun, 29 Aug 2021, 02:11 vaubaehn, @.***> wrote:

For me this is all clear (as it was also before). But I don't know, if some users will still struggle...

Ok, I decided to continue procrastinating from my work that I would need to do, and to continue on this. In general I like, what you wrote, I will try to add/rephrase some parts for even more clarity:

Update With version 2.8 the behaviour of the certificate validity check has changed. Entry rules for travellers have been set up by all EU member countries, but until today only a few countries provided technically translated rules which Corona-Warn-App can make use of. Before version 2.8, when there were no technical rules available for a certain country, after a check Corona-Warn-App displayed "Certificate Valid" and explained in the following text, that for this country no rules were provided/could have been checked. "Certificate Valid" referred to "Technical Validity" in this case [insert FAQ LINK here]. However, we understood that this result may be misinterpreted, for example when an incomplete vaccination was checked, the wrong conclusion could have been drawn, that this certificate warrants entry without restrictions. We clarified [sic. I was spelling it wrong before] this since version 2.8: In case a member country did not provide technically translated rules for Corona-Warn-App, the headline now states "Certificate Cannot Be Validated", whereas the following texts stayed the same. This does not mean that your certificate is not valid or that you are not allowed to enter the country you checked your certificate against. In this case, please verify your status with information that you find in https://reopen.europa.eu/. Further [SPELLING] information on this change can be found in the second paragraph [SPELLING] of this blogpost: [LINK]

It's a bit longer, but maybe hopefully now clear for everyone?

Thanks for your support! Need to return to my work now...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/corona-warn-app/cwa-documentation/issues/678#issuecomment-907706289, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABR7PIKMHM2WBDYJ3GU5BTT7F3JXANCNFSM5BOYRJCQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Jo-Achim commented 3 years ago

Just in case...

@vaubaehn I think the proposed solution b) would be the better one. The countries without "business rules in PROD" may additional be grayed out for identification.

The complete hiding of countries without "business rules in PROD" (a)) would only lead to the unnecessary question whether the country list is complete or why countries are missing.

I still believe that 'non-verifiable countries' should only be grayed out for labeling! Filtering out in the sense of 'not showing countries that cannot be checked' only raises questions about the completeness of the list of countries, etc., confuses more than clarity and does not improve the result.

mtwalli commented 3 years ago

Please keep in mind that users are comparing to CovPass App also. CovPass is behaving as CWA prior v2.8.0 .

vaubaehn commented 3 years ago

@mtwalli Do you mean it would be good to include differences in CWA/CovPass in an explaining FAQ text? In general, it would be good to involve CovPass devs in this issue here, imho

mtwalli commented 3 years ago

Yes, we need to explain to them why there is inconsistency between the two Apps. I agree, we need to communicate that to CovPass team

Ein-Tim commented 3 years ago

FYI: I opened https://github.com/Digitaler-Impfnachweis/covpass-ios/issues/42 for this.

vaubaehn commented 3 years ago

Regarding CovPass, for FAQ I could suggest this additional paragraph below the suggested 'update paragraph':

Why does the CovPass app show a different result when I check entry rules for a country that did not provide technically translated entry rules yet? The validity check for entry rules was first introduced in Corona-Warn-App with version 2.6.1. When this validity check was developed for CovPass, Corona-Warn-App's result screen served as a template for CovPass' current implementation. We are in contact with developers of CovPass to align the results of both apps.

How is this?

Ein-Tim commented 3 years ago

I will take all your feedback into account and provide a PR for this later today.

vaubaehn commented 3 years ago

@MikeMcC399 I would be very happy for your critical review on all proposed texts for FAQ, if you like and have the time. And if you could enhance style/expression for the English parts, would also be great, imho

Ein-Tim commented 3 years ago

I suggest to use https://github.com/corona-warn-app/cwa-website/issues/1682 for discussion about the FAQ entry. We would have to decide if it wouldn't make more sense to create a new FAQ entry for this.

MikeMcC399 commented 3 years ago

@vaubaehn

I would be very happy for your critical review on all proposed texts for FAQ, if you like and have the time. And if you could enhance style/expression for the English parts, would also be great, imho

I'm going to be busy next working week. I will try to fit in feedback as I can.

Ein-Tim commented 3 years ago

@vaubaehn

Can this be closed now with the FAQ entry https://www.coronawarn.app/en/faq/#dcc_no_rules published?

vaubaehn commented 3 years ago

@vaubaehn

Can this be closed now with the FAQ entry https://www.coronawarn.app/en/faq/#dcc_no_rules published?

@Ein-Tim @MikeMcC399 @dsarkar I'm still a bit unhappy, because the new FAQ entry is not consitently used/pointed to by the Google Play Store support team although this question still pops up from time to time, and CovPass is still behaving differently wrong... Maybe the explanation inside the app can still be enhanced? But this may not be covered satisfyingly from this issue here... So, what would you suggest - leaving it open here as long as the "core problem" (confused users) is still visible, and we could add more enhancement requests here? Or closing and opening more specific new issues?

MikeMcC399 commented 3 years ago

@vaubaehn I would suggest closing this issue and if necessary opening specific further issues, since the original expectation:

"CWA must not state that DCC for the CoA is valid, even no rules have been checked."

has been met.

The app now says "Your certificate could not be validated".

I opened a related issue https://github.com/corona-warn-app/cwa-documentation/issues/705 "Certificate validation translation error for no Business Rules" for this area, as you know. I'm afraid there hasn't been any response from the Open Source Team to this issue so far.

Ein-Tim commented 3 years ago

FYI: CovPass now also shows "Zertifikat nicht prüfbar" if there are no rules available.

Checked under iOS with CovPass version 1.10.

vaubaehn commented 3 years ago

@MikeMcC399 Thanks for your feedback and @Ein-Tim thanks for the information!

I also tried with latest CovPass for Android and can confirm that countries without machine-readable rules are now also displayed as "nicht prüfbar". With this update, personally I prefer CovPass UI over CWA's UI in that special point.

So, the "But in CovPass it's still valid" discussions should be over for now, and agreeing to Mike's feedback I'm closing this issue here again.

Maybe we meet here once more? I hope not 😉 Thanks everybody!