corona-warn-app / cwa-app-android

Native Android app using the Apple/Google exposure notification API. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
2.44k stars 496 forks source link

Cause 9002: file is not a database: , while compiling: select count(*) from sqlite_master; #642

Closed raykov closed 4 years ago

raykov commented 4 years ago

Describe the bug

I'm installing the app. The next day it stops to work and starts crashing on start. Sometimes it's getting an error

Cause: 9002 Something went wrong. file is not a database: , while compiling: select count(*) from sqlite_master;

This happened twice already. After first one I've removed and reinstalled app.

Expected behaviour

I expect the app to work longer then one day ;)

Steps to reproduce the issue

There are no any specific steps to reproduce. I'm just using the app.

Technical details

Huawei P20 Pro. (CLT-L29) EMUI version 8.1.0 Android version 8.1.0

Additional context

Photo on 18 06 20 at 16 04


Internal Tracking Id: EXPOSUREAPP-1851

Annie-G commented 4 years ago

@Annie-G Again, thanks for your feedback. This is, because for two days (today-14 days, today) the 14-day-packages to check for exposure encounters have been downloaded and checked. Anyhow, the relevant last 14 days should have been checked for encounters now. The expected side effect should be, that CWA is now not showing an undefined risk state (nicht "Risiko konnte nicht ermittelt werden"), but a (hopefully) green state. Is that correct? And "days active" should most likely be 14 of 14 days active?

Exactly. 😊

vaubaehn commented 4 years ago

@Annie-G The expected side effect should be, that CWA is now not showing an undefined risk state (nicht "Risiko konnte nicht ermittelt werden"), but a (hopefully) green state. Is that correct? And "days active" should most likely be 14 of 14 days active?

Exactly.

Cool :+1: Enjoy, and stay healthy!

thomasaugsten commented 4 years ago

@vaubaehn No update yet but we will verified this as well from our side. Our current information is only b) is deleting the stored TEKs/RPIs.

vaubaehn commented 4 years ago

@vaubaehn No update yet but we will verified this as well from our side. Our current information is only b) is deleting the stored TEKs/RPIs.

@thomasaugsten thanks a lot. that would be great to get rid of the insecurity caused by the cleared logs. :+1:

alois31 commented 4 years ago

After ~a month without this bug, it occcurs to me again every day or so (P8 lite). From reading the comments, it appears to me that the cause of this bug is most likely corruption of the encrypted database. Maybe backing up the database every day or so, before writing, and restoring that backup in case corruption is detected on startup would help?

MikeMcC399 commented 4 years ago

@thomasaugsten

@vaubaehn No update yet but we will verified this as well from our side. Our current information is only b) is deleting the stored TEKs/RPIs.

I tried CLEAR DATA on CWA 1.2.1, under Android 8 with Exposure Notification 16203314000 as in a) from this thread

a) deleting data and cache via Android > settings > apps > CWA > storage

thereafter the Exposure checks (Settings > Google > COVID-19 Exposure Notifications) were unaccessible. After restarting CWA and running through the onboarding sequence, including ACTIVATE EXPOSURE LOGGING, the old Exposure check history was lost and was replaced by a new history. (I had described this last month in https://github.com/corona-warn-app/cwa-app-android/issues/934#issuecomment-671272098 as well, and I repeated the test today.)

If the history is deleted, then I would be wary about believing that the stored Temporary Exposure Keys (TEK) and Rotating Proximity Identifiers (RPI) are preserved, however without being able to view the TEKs and RPIs to check, anything is possible. Or is there some way (without having a rooted device) of checking whether they are preserved or not?

vaubaehn commented 4 years ago

If the history is deleted, then I would be wary about believing that the stored Temporary Exposure Keys (TEK) and Rotating Proximity Identifiers (RPI) are preserved, however without being able to view the TEKs and RPIs to check, anything is possible. Or is there some way (without having a rooted device) of checking whether they are preserved or not?

The only way to get that checked for collected RPIs without root, is when you had any exposure encounter in the past 14 days (i. e., encounter with low or high risk), delete CWA data, re-setup like discribed above. If the encounter was still visible, would indicate that collected RPIs are not deleted. But no information about own TEKs, though.

By the way, thanks very much for your engagement, @MikeMcC399 ! I find your contributions very helpful.

Lotte85 commented 4 years ago

Same here, Sony XZ2 compact, Android 10.0.0 Its crashing when ever I want to open the App after running perfectly since installing it. No "9002" but the log Shows SQliteCompiledSql.Java as source file and net.sqlcipher.database.SQliteCompiledSql as source class. I will try what you told Annie-G

Lotte85 commented 4 years ago

Same here, Sony XZ2 compact, Android 10.0.0 Its crashing when ever I want to open the App after running perfectly since installing it. No "9002" but the log Shows SQliteCompiledSql.Java as source file and net.sqlcipher.database.SQliteCompiledSql as source class. I will try what you told Annie-G

Worked for me, too ☺️ I'll report how long it lasts ;)

Thanks!

thomasaugsten commented 4 years ago

We double check with Google and they are not plan to store the keys after the last Covid19 App is deleted. Because is not aware after deleting he has to remove data in the Android settings. This mean a save way to reset the app is: Android > Settings > Apps > CWA > Storage > delete data/cache

We discuss how we can help to user to restore the app when the DB is failing

vaubaehn commented 4 years ago

Hi @thomasaugsten ,

We double check with Google and they are not plan to store the keys after the last Covid19 App is deleted. Because is not aware after deleting he has to remove data in the Android settings. This mean a save way to reset the app is: Android > Settings > Apps > CWA > Storage > delete data/cache

We discuss how we can help to user to restore the app when the DB is failing

thanks a lot for your reply! If I understand everything right, it's rather good news!

I'll try to sum up the facts here, please check if everything is correct:

Collected RPIs / own TEKs are only deleted, when... a) CWA is uninstalled, and it was the only/last tracing app on the device b) when the user manually deletes RPIs/TEKs via Android > settings (>general || > apps) > Google > COVID-19-Notifications > "Zufallskennungen löschen"

In all other cases, RPIs / TEKs are preserved.

When the user deletes CWA data via Android > settings > apps > CWA > storage > data/cache, then...

as the most important data... c) the history of past checks from Android > settings (>general || > apps) > Google > COVID-19-Notifications (ENF) is erased d) everything related to a registered QR-code (token) is erased -> no test result notification via app possible, e) days of exposure logging being active are erased -> "risk status not possible, not long enough acitve"-message,

as the least important data... f) everything else, that is stored via EncryptedSharedPreferences, but which can easily be re-setup (via onboarding), is erased g) the keyCache-db (sqlite), that holds data that might easily retrieved again by ENF-api, is erased.

Is that correct?

@thomasaugsten and @svengabr , would it then make sense to provide short instructions in the FAQ for affected users, on how to reset CWA data via OS, at least as a first aid measure, until a better solution is provided trough the app? The biggest trade-off would result, if someone registered a test before (now needs to get the result via phone/mail). But at least exposure logging/general app usage could continue. And with the date-change-trick (https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-685008010) CWA would most likely look like before...

How is your opinion about it?

thomasaugsten commented 4 years ago

This looks good we can add this to the FAQ. I think one day back instead of 14 should also work here.

vaubaehn commented 4 years ago

@thomasaugsten and @svengabr

This looks good we can add this to the FAQ.

any support wanted? I could translate it to human language...

Anyway, maybe it's better to clarify with all devs first. If it's welcome, would volunteer to create a text.

svengabr commented 4 years ago

@vaubaehn

Of course, support is always welcome! If you want to volunteer to change the FAQ, please open an issue on the cwa-website repository explaining the change that should be performed on the website. You can also mention our issue here.

Cheers!

mhellmeier commented 4 years ago

Same problem here but on a Xiaomi Mi Mix 2S device (so no Huawei), not rooted. Got this problem for the second time!

Deleting the app data and working with the temporary time-shift trick worked for me.

Hope to see a fix, soon! If you need more information, stack trace etc. I am ready to help!

Annie-G commented 4 years ago

@Annie-G The expected side effect should be, that CWA is now not showing an undefined risk state (nicht "Risiko konnte nicht ermittelt werden"), but a (hopefully) green state. Is that correct? And "days active" should most likely be 14 of 14 days active?

Exactly.

Cool 👍 Enjoy, and stay healthy!

Hi! The Issue reappeared and I had to do the delete app data thing again after a few days. Now, about a week after, I have error #39508. This error I remember appeared before, but only when I opened the app at night between midnight and 2am. Now I have error #39508 in daytime, I wonder if it is connected with the crash-issue or with me having deleted app data.

alois31 commented 4 years ago

@Annie-G Errror 39508 is a quota error from Play services, that one can happen when resetting the app. It should go away tomorrow.

alois31 commented 4 years ago

I think I have a semi-reliable (at least on P8 lite) reproducer right now that should work ~twice a day (before getting 39508).

  1. Enable the data usage meter in the status bar and make sure that no background downloads are running (not directly related, but helpful later).
  2. Make sure that CWA will update when opening (e.g. by following the procedure in https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-685008010, except the last step).
  3. Open CWA, which will now download the keys of infected people. Wait until the download is complete (look at the data usage meter).
  4. Quickly close CWA by swiping it up in the overview. Just pressing the home button is not enough. This should happen before CWA has finished the risk update.
  5. Open CWA again, you should see this bug now.

In order to prevent this procedure (which may accidentally happen in the wild) from working, I think some form of https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-685539628 would need to be implemented, or sqlcipher needs to be taught to treat decryption errors as corruption and not an exception (I think the journaling in SQLite would in principle be enough to recover in this case).

On the other hand, the common case of this bug might be when the background update is not prioritized and is killed at an unfortunate moment. In this case, it might be enough to not even attempt the background update if it is not prioritized (which might also fix some (most?) cases of #1121 (a.k.a. #1053?)). It might also be a good idea to tell the user to keep CWA open while it performs a foreground update.

I think in order to make a better judgement, it would be helpful to know from the people who have experienced this bug (if they still remember):

Annie-G commented 4 years ago

I think I have a semi-reliable (at least on P8 lite) reproducer right now that should work ~twice a day (before getting 39508).

1. Enable the data usage meter in the status bar and make sure that no background downloads are running (not directly related, but helpful later).

2. Make sure that CWA will update when opening (e.g. by following the procedure in [#642 (comment)](https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-685008010), except the last step).

3. Open CWA, which will now download the keys of infected people. Wait until the download is complete (look at the data usage meter).

4. Quickly close CWA by swiping it up in the overview. Just pressing the home button is not enough. This should happen before CWA has finished the risk update.

5. Open CWA again, you should see this bug now.

In order to prevent this procedure (which may accidentally happen in the wild) from working, I think some form of #642 (comment) would need to be implemented, or sqlcipher needs to be taught to treat decryption errors as corruption and not an exception (I think the journaling in SQLite would in principle be enough to recover in this case).

On the other hand, the common case of this bug might be when the background update is not prioritized and is killed at an unfortunate moment. In this case, it might be enough to not even attempt the background update if it is not prioritized (which might also fix some (most?) cases of #1121 (a.k.a. #1053?)). It might also be a good idea to tell the user to keep CWA open while it performs a foreground update.

I think in order to make a better judgement, it would be helpful to know from the people who have experienced this bug (if they still remember):

* Did you have prioritized background activity enabled?

* Did you close CWA while it was performing an update?

Priority background activity is enabled. As far as I know, I have not closed CWA while it was performing an update.

alois31 commented 4 years ago

Priority background activity is enabled. As far as I know, I have not closed CWA while it was performing an update.

What is your phone model? Is CWA protected and ignore optimizations enabled on CWA?

Annie-G commented 4 years ago

Priority background activity is enabled. As far as I know, I have not closed CWA while it was performing an update.

What is your phone model? Is CWA protected and ignore optimizations enabled on CWA?

Running the App on Lenovo Moto Z (XT1650-03) Build OPL27.76-71-2-3, Android 8.0.0. Settings of CWA say: Battery optimization: Not optimized

alois31 commented 4 years ago

@Annie-G Information seems relatively sparse for Lenovo devices, but some people suggest that they tend to kill even unoptimized apps unless they show a padlock icon in the app drawer (I don't know if that applies to background services though). Try setting CWA to optimized and then back to unoptimized again.

If you see this bug again, could you check whether the procedure from https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-691938723 makes the bug appear a second time?

benediktrauch commented 4 years ago

Got the error after updating Pixel 2 XL to Android 11.

Screenshot_20200915-133538.png

vaubaehn commented 4 years ago

@thomasaugsten and @d4rken , please note comment of @benediktrauch above: do system upgrades alter/change the Android master key? If yes, current implementation of db-password inside EncryptedSharedPreferences has another important flaw in design, as many people will start to upgrade to Android 11 from now on. @d4rken , you already had a change of design in mind, if I remember right?

vaubaehn commented 4 years ago

Hi @benediktrauch , see here => https://github.com/corona-warn-app/cwa-app-android/issues/1053#issuecomment-690614975 - this will most likely solve the problem for you for now. Cheers

benediktrauch commented 4 years ago

Hi @benediktrauch , see here => https://github.com/corona-warn-app/cwa-app-android/issues/1053#issuecomment-690614975 - this will most likely solve the problem for you for now. Cheers

Yes.

Changed date 14 days back. Deleted app data an cache. Set date to yesterday opened app fetched keys and changed date to today. Now it's works again an shows "dauerhaft aktiv".

Thanks.

Neuwessi11 commented 4 years ago

Hallo in die Runde!

Ich habe in den letzten Wochen hier mitgelesen, aber leider wegen meiner limitierten technischen und Englischkenntnisse die wichtigere Hälfte nicht verstanden.

Kann mir bitte nochmal jemand eine deutsche Zusammenfassung des aktuellen Stands zu den folgenden Fragen geben?

  1. Was kann man tun, wenn dieser Fehler aufgetaucht ist? Die gesammelten Kontaktschlüssel sollen bitte bei der Fehlerlösung erhalten bleiben, also nicht gelöscht werden.
  2. Wenn man das gemacht hat, was (so jedenfalls meine Hoffnung) unter 1. vorgeschlagen wurde: Wird der Fehler damit behoben oder hat man dann nur für ein paar Tage Ruhe?
  3. In der Version 1.3.0 scheint das Problem ja noch nicht behoben zu sein. Besteht Hoffnung auf eine baldige Lösung des Problems mittels Update?
  4. Ist denn inzwischen die (softwareseitige) Ursache des Problems zumindest erkannt?

Vielen Dank für die Hilfe!

Neuwessi

vaubaehn commented 4 years ago

Hallo in die Runde!

Hallo @Neuwessi11 !

Ich habe in den letzten Wochen hier mitgelesen, aber leider wegen meiner limitierten technischen und Englischkenntnisse die wichtigere Hälfte nicht verstanden.

Kann mir bitte nochmal jemand eine deutsche Zusammenfassung des aktuellen Stands zu den folgenden Fragen geben?

1. Was kann man tun, wenn dieser Fehler aufgetaucht ist? Die gesammelten Kontaktschlüssel sollen bitte bei der Fehlerlösung erhalten bleiben, also nicht gelöscht werden.

Du kannst die folgende Anleitung (auf deutsch) befolgen: https://github.com/corona-warn-app/cwa-app-android/issues/1053#issuecomment-690614975 Nach dem aktuellen Stand unserer Kenntnisse werden die gesammelten Kontaktschlüssel dabei nicht gelöscht.

2. Wenn man das gemacht hat, was (so jedenfalls meine Hoffnung) unter 1. vorgeschlagen wurde: Wird der Fehler damit behoben oder hat man dann nur für ein paar Tage Ruhe?

Es ist leider nicht ganz unwahrscheinlich, dass der Fehler nach einiger Zeit wieder auftritt. Bei einigen dauert es viele Wochen, bei anderen passiert es schon nach wenigen Tagen.

3. In der Version 1.3.0 scheint das Problem ja noch nicht behoben zu sein. Besteht Hoffnung auf eine baldige Lösung des Problems mittels Update?

Nach der Durchsicht des Programm-Codes und einiger Kommentare von Entwicklern (special thanks to @d4rken !!!), ist zu hoffen, dass sich bald eine Besserung zeigen wird. Die Ursachen des Problems können vielfältig sein, und haben ihre Ursrünge außerhalb der Corona-Warn-App. Die Entwickler versuchen mit verschiedenen Maßnahmen, den Problemen entgegen zu wirken. Die Änderungen sind aber nicht ganz unheikel, und müssen zunächst darauf überprüft werden, das möglichst alle Geräte davon profitieren und nicht neue Probleme eingeführt werden. Nach meiner persönlichen Auffassung ist leider nicht mit einer schnellen Lösung zu rechnen. Aber für eine konkrete Aussage dazu müsste einer der Entwickler hier Stellung beziehen.

4. Ist denn inzwischen die (softwareseitige) Ursache des Problems zumindest erkannt?

Es gibt möglicherweise viele unterschiedliche Ursachen, und wie wahrscheinlich jede einzelne zu dem Problem beiträgt, wird hier ganz unterschiedlich diskutiert. Ich persönlich bin mir recht sicher, dass vor allem das 'Killen' von CWA durch die Akku-Optimierungen zumindest in der Anfangszeit von CWA eine bedeutende Rolle gespielt hat, und eher nur bei Geräten weniger Hersteller aufgetreten war. Mittlerweile sind weitere Geräte/Marken hinzugekommen. Die Akku-Optimierungen oder das unerwartete Beenden der CWA können immer noch eine Rolle spielen, aber auch Upgrades auf eine neuere Betriebssystem-Version, Fehler im Betriebssystem, Inkompatibilitäten zwischen dem Betriebssystem und den geforderten/angewandten Verschlüsselungstechniken durch CWA. Anfangs wurde vermutet, ob Geräte vor allem chinesischer Hersteller möglicherweise versuchen, (automatisiert) Einsicht in die geschützten Daten zu erhalten und dabei (unabsichtlich) die verschlüsselten Datenbanken 'kaputtmachen'. Auch wenn man das vielleicht nicht vollständig ausschließen kann, halte ich persönlich es für eher unwahrscheinlich. Aufgrund der Vielzahl möglicher Ursachen und der Schwere des auftretenden Problems könnte ich mir vorstellen, dass die Entwickler über eine Änderung des Designs der Datenspeicherung nachdenken. Aber das ist nur Spekulation, dazu müsste wieder einer der Entwickler hier Stellung nehmen.

Druidikaaa commented 4 years ago

Habe die App seit Release und sie hat vermeintlich auch funktioniert. Anfangs habe ich sie noch ab und zu geöffnet. Mittlerweile wische ich nur noch die wöchentliche Info über den Status weg.

Heute habe ich die App Mal wieder geöffnet und musste mich durch die Startdialoge tippen, was an sich ja schon bedenklich ist. Beim Aktivieren der Risikoermittlung bekomme ich ebenfalls die Meldung aus diesem Ticket. Beenden erzwingen hat nichts geholfen. Neu installiert habe ich noch nicht.

Pixel 2 XL, Android 10 (Morgen installiere ich aber Android 11)

vaubaehn commented 4 years ago

Hallo @Druidikaaa , dann installiere doch bitte zunächst Android 11. Sehr wahrscheinlich wird die App beim Öffnen dann noch immer den gleichen Fehler zeigen, oder direkt abstürzen. In dem Falle folge bitte dieser Anleitung: https://github.com/corona-warn-app/cwa-app-android/issues/1053#issuecomment-690614975 Bitte nicht neu installieren, weil dann alle Begegnugen gelöscht werden. Melde dich doch dann bitte kurz, ob bei dir wieder alles funktioniert. Vielen Dank!

d4rken commented 4 years ago

While I still can't reproduce it reliably, I've got a theory now that I consider plausible enough to be able to write a mitigation for this.

TL;DR: The core idea is to upgrade to security-crypto-1.0.0-rc03, catch KeystoreException and retry the initialization of our encrypted storage with an exponential backoff mechanism.

In detail: We upgrade to "security-crypto-1.0.0-rc03" from rc02 which makes the EncryptedSharedPreferences internally use Tink 1.4, here Google's changelog mentions

Bugfixes: Tink update should gracefully handle AndroidKeyStore concurrency failures.

Sadly they don't link directly to a commit that fixes this to better understand it.

There are over 250 commits between Tink 1.3.0 and Tink 1.4.0, but it contains mitigation for an issue in Tink that seems awfully close to our issue.

That issue pairs with this issue ticket on Google's tracker for the security-crypto library.

Specifically they mention this problematic behavior

We did some more research on our side and we are certain that most of these errors are caused by the keystore being busy, which will give an occasional error. This happens more often in the background because services that all wake up at the same time might be trying for keystore access. I will update this bug when it goes live. Appreciate the patience all.

So what's likely going on is the a combination of different issues:

  1. The Android Keystore may return a "Busy" Error on some devices, because it only allows a limited amount of concurrency.
  2. Tink 1.3 assumes a permanent encryption error when encountering this "Busy" exception and will just rewrite the current encryption keys (losing the original).
  3. EncryptedSharedPreferences is none the wiser and will attempt to use the new keys that Tink 1.3 just created due to the "Busy" error.
  4. This leads to us no longer having the original password we used to encrypt the database. From the databases perspective, not being a valid database or not correctly decrypting it, is the same just looks like random data.

By upgrading to "security-crypto-1.0.0-rc03", we get Tink 1.4, which in turn will no longer overwrite the encryption keys on congestion, but instead throw a recognizable exception. We can then detect that exception and perform retry with exponential backoff. If I'm right, this should already fix a lot of cases.

This being a congestion based issue would also correlate well with the issue being more prevalent on low end devices (Huawei P8 Lite) which may have an increased chance of apps running for longer in parallel. The Pixel 2 upgrade error also fits, because update->reboot->many apps initialize and potentially try to access the keystore at once.

alois31 commented 4 years ago

@d4rken Awesome finding, this seems to explain #1121 as well if I'm not completely mistaken. Do you have a theory under which circumstances #1121 vs. this bug happens, as I honestly don't see why EncryptedSharedPreferences would give out garbage data (as it uses authenticated encryption).

I also want to add that your theory may explain why my procedure from https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-691938723 works: The congestion is more likely to happen when I use the phone, and "being quick enough" in step 4 may very well be "the update doesn't finish because the db is already broken".

svengabr commented 4 years ago

The developer conversation a few days ago shows the same

I think I now know what happens.

Besides evaluating to remove encryption on unnecessarily encrypted data, we should do the following:

Upgrade to "security-crypto-1.0.0-rc03" for our next 1.4.0 RC or 1.5.0, catch KeystoreException and retry with an exponential backoff.

d4rken commented 4 years ago

The developer conversation a few days ago shows the same

I think you just quoted me :wink:

arianvp commented 4 years ago

The problem happened to me after scanning the test result QR code and reopening the app.

I updated to android 11 a few weeks ago (Pixel 2) and it was working fine; but since today it doesn't work.

I closed the dialog, but not the app won't open at all . "App keeps crashing"

I reset the app (Cleared data); but now I can not rescan my QR code as "it has been used already".

:/

daimpi commented 4 years ago

@arianvp Yes this is really unfortunate… the QR codes are one time use for privacy reasons:

Hi unfortunately this is not possible. We have to ensure only one phone can scan the qr code and submit his diagnoses keys. We are not allowed to save an ID of the phone this is the reason why the qr code is only valid for one scan. Please wait for the normal way (a letter via post mail or Gesundheitsamt calls you) of the test lab to get your result.

(link)

The situation with the non-accessible database will hopefully be resolved once CWA 1.4 is released.

arianvp commented 4 years ago

Question. Did my diagnoses keys already upload after scanning but before testing positive? E.g. people will still be notified if I test positieve in the next few days even though I won't get the notification in the app myself anymore?

daimpi commented 4 years ago

Question. Did my diagnoses keys already upload after scanning but before testing positive?

Your diagnosis keys (DKs) will only be uploaded once you get a positive result back and then decide to share your keys.

But: If you "only" cleared your data without uninstalling CWA your DKs might still be on your phone:

According to the information currently available #642 (comment) , the codes of encounters with other users stored on your phone are retained, so that your risk for the last 14 days can be determined again the next time you start the app.

(link)

Once you get your result back and if it is actually positive, you can call the tele-TAN hotline and request a TAN with which you then can upload your DKs.

arianvp commented 4 years ago

Thanks for the info. Also happy to hear we've singled out what the cause is of the issue and a fix is on the way :)

vaubaehn commented 4 years ago

FYI @d4rken : NHS Covid App is using a similar approach to retry on EncryptedSharedPreferences like you implemented. This is how new Tink exceptions look like: https://github.com/nhsx/covid-19-app-android-ag-public/issues/13 and https://github.com/nhsx/covid-19-app-android-ag-public/issues/22 . Happy to see https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-697603175 being validated already :+1:

d4rken commented 4 years ago

Thanks for the links.

NHSX seems to use more specific (GeneralSecurityException) matching, while we retry for any exception during encryption. I wanted to err on the safe side here.

https://github.com/corona-warn-app/cwa-app-android/blob/7ff340f673103b92ce1e37fe383fec14a6afa4e1/Corona-Warn-App/src/main/java/de/rki/coronawarnapp/util/security/EncryptedPreferencesFactory.kt#L31

I think :thinking: this should only be an issue if there is any other reason besides the Tink issue causing exception regularly, currently I don't think there is any. Worst case, there is another exception, then we delay surfacing that exception for 15s. More specific matching is always good though, let's monitor the effect of this mitigation and then potentially make the catch more specific. The stacktraces in the linked tickets will definitely help with that.

bobbolous commented 4 years ago

Same issue here with LineageOS 16 (Android 9) on a Samsung S5 (SM-G900F). The app did however open and showed up as if i would open it the first time (tutorial etc). I did reinstall the app and now it works again. But I lost the last 2 weeks of tracking.

mh- commented 4 years ago

In order to not loose the last 2 weeks of tracking:

daimpi commented 4 years ago

@mh- good idea 👍

But you probably don't even have to delete CWA to restore it: @vaubaehn wrote a manual on how to recover such states and it seems to suffice to delete cache and data: https://github.com/corona-warn-app/cwa-app-android/issues/1053#issuecomment-690615473. In this case you don't have to delete CWA and therefore there's also no need to install another intermediate app 🙂.

eliasweingaertner commented 4 years ago

Just bumped into the same issue on a Xiaomi Poco F1 on MIUI Global 11.0.9. Had to reset all app data to make it work.

patrickjDE commented 4 years ago

Just got the same error message and behaviour. Xperia xz1 compact, Android 9, latest official build 47.2.A.11.228

It happened when I opened the app for the first time after my battery had run dry yesterday. As far as I can tell this has not yet happened after a "normal" shutdown/restart or reboot, but I had it happen after experiencing something similar to #597 (same error, but it was triggered by fingerprint unlocking, not by manually turning BT or GPS off).

Hope this info helps with further digging into the root causes of the issue.

553755 commented 4 years ago

I've got this error for the second time now:

image

Both times I deleted app cache and data, changed system date of the phone back to an earlier date, then the app start working again. Now the app shows me no risks, but I know there was some low risk contacts just yesterday showing up in the app. Will the app still send me a notification about a high risk contact after this person validate her test, but not showing up low risk contacts which was probably just stored at the phone itself?

As this is the second time I have to reset the app (last time was about 10 days ago) this is very annoying. I own a Sony XZ2 Compact with Android 10.

daimpi commented 4 years ago

@553755

Both times I deleted app cache and data, changed system date of the phone back to an earlier date, then the app start working again. Now the app shows me no risks, but I know there was some low risk contacts just yesterday showing up in the app. Will the app still send me a notification about a high risk contact after this person validate her test, but not showing up low risk contacts which was probably just stored at the phone itself?

The way the risk calculation currently works is that CWA downloads Diagnosis Keys (DKs) and parameters from the server and hands them over to Google's Exposure Notification Framework (ENF). ENF then compares the rolling proximity identifiers (RPIs) it recorded over the last 14 days with the DKs to see whether there is a match. In case of a match it will also decrypt the associated encrypted metadata (AEM) to figure out how intense the contact was and depending on the parameters it got from CWA this will then either be classified as a "increased-risk/red" exposure or a "low-risk/green" encounter. So it shouldn't matter whether you had a green or red encounter, either of them should show up again after resetting CWA data (as long as you don't re-install/uninstall CWA, b/c if the last ENF app is uninstalled from your phone ENF will delete all the keys it saved).

What could have happened is that the encounter which you had was just on the brink of leaving the 14 day storage window the last time you checked before you reset your CWA data and after the reset the encounter was older than 14 days, therefore phased out by ENF and the match disappeared.

Regarding the underlying problem: https://github.com/corona-warn-app/cwa-app-android/pull/1235 hopefully fixes this issue. This PR is part of CWA 1.5 which should hit the play store tomorrow.

553755 commented 4 years ago

@daimpi

What could have happened is that the encounter which you had was just on the brink of leaving the 14 day storage window the last time you checked before you reset your CWA data and after the reset the encounter was older than 14 days, therefore phased out by ENF and the match disappeared.

How likely would it be that all of them expired at the same day, just when the app stopped working correctly? I guess that's a very unlikely event. So I will look forward to to CWA 1.5.

daimpi commented 4 years ago

@553755

How likely would it be that all of them expired at the same day, just when the app stopped working correctly? I guess that's a very unlikely event.

Hmm… good question… If you met e.g. a group of ppl on a specific day which then later tested positive, this would exactly play out like this, i.e. all encounters would disappear on the same day.

Do you by chance have an EN log from before resetting the data? And what does your app and EN log now say wrt the date that the last checks were performed?

@vaubaehn I'm wondering whether the procedure which changes the clock on the phone could impact the keys stored on the device. We know e.g. that keys older than 14 days will be phased out by ENF… I wonder if there is also a mechanism which deletes keys which have their validity-date in the future, as they could potentially seen as invalid 🤔.