search

corona-warn-app / cwa-app-android

Native Android app using the Apple/Google exposure notification API. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
2.44k stars 496 forks source link

Cause 9002: file is not a database: , while compiling: select count(*) from sqlite_master; #642

Closed raykov closed 4 years ago

raykov commented 4 years ago

Describe the bug

I'm installing the app. The next day it stops to work and starts crashing on start. Sometimes it's getting an error

Cause: 9002 Something went wrong. file is not a database: , while compiling: select count(*) from sqlite_master;

This happened twice already. After first one I've removed and reinstalled app.

Expected behaviour

I expect the app to work longer then one day ;)

Steps to reproduce the issue

There are no any specific steps to reproduce. I'm just using the app.

Technical details

Huawei P20 Pro. (CLT-L29) EMUI version 8.1.0 Android version 8.1.0

Additional context

Photo on 18 06 20 at 16 04

Internal Tracking Id: EXPOSUREAPP-1851

mikehaertl commented 4 years ago

For everyone having this issue: It's common habit on Github to click the "Thumbs up" button on the issue report on top. This makes it easier for the developers to see how many people are suffering from a bug. This could also give an issue more weight and increase the chance that it's fixed soon. At least that's how most OSS projects handle it - this here may be a little different, though.

To make this clear: I'm not related to the project, just saying. And yes, I had this issue too on a Huawei P8 (installed some weeks ago, error appeared today) and reinstalling the app seems to fix it.

vaubaehn commented 4 years ago

Hi @raykov , sorry to bother you again. Can you do me a favor, and check whether the feature 'geschützte Apps' (protected Apps?) in settings > 'Akkumanager' (battery manager?) as described here https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-673567727 by @marcelm is not present anymore in your Android version, or at least CWA is enabled as protected App? That would help a lot, thanks in advance!

Martin-Luft commented 4 years ago

@uweRem the error occurs regularly and I had to reinstall the app everytime. I have a 128 GB SdCard in my Honor and I use the cool feature that the phone uses the SdCard as the default storage :)

vaubaehn commented 4 years ago

@Martin-Wegner

the error occurs regularly and I had to reinstall the app everytime. I have a 128 GB SdCard in my Honor and I use the cool feature that the phone uses the SdCard as the default storage :)

I consider this to be an extremely important information! If restrictive energy policies are in effect, this might lead to a late initialization of SD card storage, or adds in the risk for CWA's database not being closed properly, which might lead to a corrupt database file. All this can result in a crash of CWA upon opening. Can you consider to reinstall CWA on internal storage, and see, if it mitigates your problem?

@jakobmoellersap : Maybe you want to consider this: https://github.com/corona-warn-app/cwa-app-android/issues/579#issuecomment-674405103 ?

Martin-Luft commented 4 years ago

@vaubaehn this is tricky. On installation you cannot control the destination storage. And remember, the SdCard is the default storage (it is like an internal storage emulation). The app settings for the Corona App are set to internal storage by default (but the SdCard is the internal storage!?). I could change it to SdCard but this sounds very weird.

vaubaehn commented 4 years ago

@Martin-Wegner just to prevent any confusion and misunderstanding: your 128 GB SD card is an external storage card, which is replacable, right?

vaubaehn commented 4 years ago

@Martin-Wegner ...and it is emulated to behave like internal storage, right?

Martin-Luft commented 4 years ago

@vaubaehn this is correct. The default storage path is the SDCard. So all apps are installed on the SdCard by default and all apps save their data on the SdCard by default. Only the android OS itself is stored on the internal storage. The weird point is, that the storage options are set to 'internal storage' for every app you click in the Android app settings (which is only true if you interpret the SdCard as an internal storage). What can I test for you?

uweRem commented 4 years ago

On my Huawei P8 the app is installed on internal storage and the error comes up. Reinstalling the app helps for some hours. Obviously this has the same effect as deleting the data of the app under android preferences/Apps/CWP/memory/delete data (Daten löschen).

vaubaehn commented 4 years ago

@uweRem If you want to support us (the user community), would you like to try something out and report in a couple of days, whether it worked for you?

Hope, it helps for now, and I'm curious for your feedback.

vaubaehn commented 4 years ago

@Martin-Wegner if you want, you may try if the steps that I wrote to @uweRem here https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-674787145 may also work for you. But additionally to all other users, your SD card is one additional bottleneck... Looking forward for your feedback!

marcelm commented 4 years ago

I played around a bit trying to trigger the crash again. It turns out that possibly the "geschützte App" thing is not a permanent solution.

What I did:

I’ll report back in a couple of days. I cannot follow the "leave the phone on the charger as long as possible" workaround, though, as it’s not my phone.

vaubaehn commented 4 years ago

@marcelm do you remember which type of 9002 it was (there are several)?

vaubaehn commented 4 years ago

And @JoFriede , @kryops , @raykov , @uweRem , @Martin-Wegner and @marcelm , in any case thanks for replying to my question 4 days ago!

vaubaehn commented 4 years ago

@marcelm I wonder, if the setting "Geschützte App" indeed had an effect. If CWA's database was properly closed, while you were disabling "Geschützte App" -> no problem. When you then restart CWA, and close it later, it might be, that CWA's tasks are killed (more or less immediatly, because now CWA is not "geschützt" anymore) before the database gets written to disk/internal storage -> database corrupted -> 9002 sqlite or crash. Can you reproduce?

schmdan1974 commented 4 years ago

I have the same Problem on my Sony Xperia XZ1 (Android 9) Build: 47.2.A.11.228

vaubaehn commented 4 years ago

Hi @schmdan1974 , have you also been affected by issue #597 more or less shortly before error '9002: file is not a database' appeared on your device? Your response may help to locate the source of the problem '9002: file is not... ' more precisely. Thanks in advance for your reply. Kind regards!

schmdan1974 commented 4 years ago

Yes, i am also affected. My device is à Sony XZ1

vaubaehn notifications@github.com schrieb am Fr., 21. Aug. 2020, 12:46:

Hi @schmdan1974 https://github.com/schmdan1974 , have you also been affected by issue #597 https://github.com/corona-warn-app/cwa-app-android/issues/597 more or less shortly before error '9002: file is not a database' appeared on your device? Your response may help to locate the source of the problem '9002: file is not... ' more precisely. Thanks in advance for your reply. Kind regards!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/corona-warn-app/cwa-app-android/issues/642#issuecomment-678222316, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQU65NPKURHVUZ564ACL5ETSBZF65ANCNFSM4OBUNWCA .

helmutweick commented 4 years ago

After many stable weeks the error also reapperad on my Android 6.0, Huawei H60-40. Here is a summary:

After some days with CWA V1.04 the "Ursache 9002, file is not a data base" appeared error vanished after installation of CWA V1.1, then worked stable for many weeks, also as V1.2.1. But on 20.08. directly crash after start, always, also after reboot.

[bugreport-2020-08-20-23-17-30.txt] (other reason not 9002) 08-20 23:17:44.089 3948 4099 I logserver: extract_appname, forward search, appname=de.rki.coronawarnapp 08-20 23:17:44.089 3948 4099 I logserver: archive_and_send, pos=0, type=crash, output=20200820231744_crash 08-20 23:17:44.089 3948 4099 I logserver: ---copy_match_files enter!!-- 08-20 23:17:44.090 4226 4243 W System.err: java.lang.NullPointerException: Attempt to invoke virtual method 'int com.huawei.lcagent.client.LogCollectManager.getUserType()' on a null object reference 08-20 23:17:44.090 4226 4243 W System.err: at com.android.server.util.ReportTools.getUserType(ReportTools.java:86) 08-20 23:17:44.090 3948 4099 I logserver: [copy_match_files,864]: copy [/data/system/dropbox/data_app_crash@1597958264055.txt] to [/data/log/logcache/-1228392144/data_app_crash@1597958264055.txt] 08-20 23:17:44.090 3948 4099 I logserver: get_fault_appname, appname=de.rki.coronawarnapp 08-20 23:17:44.090 4226 4243 W System.err: at com.android.server.util.ReportTools.isBetaUser(ReportTools.java:73) 08-20 23:17:44.090 4226 4243 W System.err: at com.android.server.util.ReportTools.report(ReportTools.java:58) 08-20 23:17:44.090 4226 4243 W System.err: at com.android.server.util.HwUserBehaviourRecord.appExitRecord(HwUserBehaviourRecord.java:65) 08-20 23:17:44.090 3948 4099 I logserver: handle_archive_exception, BASIC_MODE 08-20 23:17:44.090 4226 4243 W System.err: at com.android.server.am.ActivityManagerService$UiHandler.handleMessage(ActivityManagerService.java:1523) 08-20 23:17:44.090 3948 4099 I logserver: remove_last_modify_file, into 1 times, file_dir: 08-20 23:17:44.090 4226 4243 W System.err: at android.os.Handler.dispatchMessage(Handler.java:102) 08-20 23:17:44.090 4226 4243 W System.err: at android.os.Looper.loop(Looper.java:150) 08-20 23:17:44.090 4226 4243 W System.err: at android.os.HandlerThread.run(HandlerThread.java:61) 08-20 23:17:44.090 4226 4243 W System.err: at com.android.server.ServiceThread.run(ServiceThread.java:46) 08-20 23:17:44.090 4226 4243 E ReportTools: This is not beta user build 08-20 23:17:44.090 3948 4099 I logserver: remove_last_modify_file, into 1 times, file_dir: 08-20 23:17:44.091 3948 4099 I logserver: get_disk_available_size, Disk_available = 2465071104 B = 2350 MB 08-20 23:17:44.091 3948 4099 I logserver: pack_files, output_path is /data/log/unzip. 08-20 23:17:44.091 3948 4099 I logserver: move_input_files, create dir [/data/log/unzip/H60-L04_H60-L04C432B860_0000000000_20200820231744_crash] 08-20 23:17:44.097 3948 4099 I logserver: handle_archive_exception, into set notify_type 08-20 23:17:44.097 3948 4099 I logserver: get_notify_mode, 1 08-20 23:17:44.097 3948 4099 I logserver: Process 4099 opened FIFO(12) for O_WRONLY 08-20 23:17:44.097 3948 4099 I logserver: notify_logcontrol, 4099 sent /data/log/unzip/H60-L04_H60-L04C432B860_0000000000_20200820231744_crash 08-20 23:17:44.097 3948 4099 I logserver: check_dir_size, dir[/data/log/coredump/] doesn't exist 08-20 23:17:44.097 3948 4099 I logserver: clean_cur_cache:999, system(rm -r /data/log/logcache/-1228392144/* > /dev/null 2>&1) 08-20 23:17:44.098 3948 3948 I logserver: process_event 08-20 23:17:44.099 3948 3948 I logserver: handle_fifo_msg, read res = 460, client pid = 4099, command = send log, data=/data/log/unzip/H60-L04_H60-L04C432B860_0000000000_20200820231744_crash 08-20 23:17:44.099 3948 3948 I logserver: handle_fifo_msg, 4099 sent /data/log/unzip/H60-L04_H60-L04C432B860_0000000000_20200820231744_crash 08-20 23:17:44.099 3948 4098 I logserver: thread_logcontrol: /data/log/logcontrol has changed. 08-20 23:17:44.099 3948 4098 I logserver: handle_notify_event, send msg [submit:trigger=0,bugtype=2,modulename=de.rki.coronawarnapp,level=1,testtype=NORMAL,path=/data/log/unzip/H60-L04_H60-L04C432B860_0000000000_20200820231744_crash,mode=1;] 08-20 23:17:44.099 3948 4098 I logserver: send_to_client, send to (9) res = 167 08-20 23:17:44.114 15684 15774 I WM-GreedyScheduler: Ignoring schedule request in non-main process 08-20 23:17:44.114 15684 15684 I Process : Sending signal. PID: 15684 SIG: 9 08-20 23:17:44.114 15743 15743 I art : Rejecting re-init on previously-failed class java.lang.Class 08-20 23:17:44.115 15743 15743 I art : Rejecting re-init on previously-failed class java.lang.Class 08-20 23:17:44.117 5397 7673 I PgedBinderListener: kstate callback type:8 value1=15684 value2=KILLED 08-20 23:17:44.118 4226 6002 E HsmCoreServiceImpl: onTransact in code is: 102 08-20 23:17:44.118 4226 6002 I MediaProcessHandler: processOp opType: 1, uid: 10008, pid: 14107 08-20 23:17:44.118 4226 6002 W MediaProcessHandler: remove target not exist, maybe the UI process: uid: 10008, pid: 14107 08-20 23:17:44.119 5397 6373 I ash : de.rki.coronawarnapp { running duration=1536 } transition to: over, reason:crash 08-20 23:17:44.119 5397 6373 I ash : remove app record pkg: de.rki.coronawarnapp

Next day app started like after new installation, but no activation possible due to "Ursache 9002 file is not a data base". Now app crashes at each start after a second, can just see screen for short.

[bugreport-2020-08-21-13-23-54.txt2] 08-21 13:22:57.034 22525 22525 I am_on_resume_called: [0,de.rki.coronawarnapp.ui.LauncherActivity] 08-21 13:22:57.257 4226 4249 I am_activity_launch_time: [0,160062039,de.rki.coronawarnapp/.ui.LauncherActivity,1383,1383] 08-21 13:22:57.291 4226 6002 I am_destroy_service: [0,190431986,6574] 08-21 13:22:57.395 4226 4878 I am_home_stack_moved: [0,0,6,6,sourceStackToFront] 08-21 13:22:57.395 4226 4878 I wm_task_moved: [25836,1,3] 08-21 13:22:57.398 4226 4878 I am_create_activity: [0,29948251,25836,de.rki.coronawarnapp/.ui.main.MainActivity,NULL,NULL,NULL,0] 08-21 13:22:57.400 4226 4878 I am_pause_activity: [0,160062039,de.rki.coronawarnapp/.ui.LauncherActivity] 08-21 13:22:57.402 4226 4878 I am_home_stack_moved: [0,0,6,6,startedActivity setFocusedActivity] 08-21 13:22:57.402 4226 4878 I wm_task_moved: [25836,1,3] 08-21 13:22:57.408 4226 4878 I am_focused_activity: [0,de.rki.coronawarnapp/.ui.main.MainActivity] 08-21 13:22:57.409 4226 5547 I am_finish_activity: [0,160062039,25836,de.rki.coronawarnapp/.ui.LauncherActivity,app-request] 08-21 13:22:57.416 22525 22525 I am_on_paused_called: [0,de.rki.coronawarnapp.ui.LauncherActivity] 08-21 13:22:57.419 4226 4865 I am_restart_activity: [0,29948251,25836,de.rki.coronawarnapp/.ui.main.MainActivity] 08-21 13:22:57.423 4226 4865 I am_focused_activity: [0,de.rki.coronawarnapp/.ui.main.MainActivity] 08-21 13:22:57.806 22525 22525 I am_on_resume_called: [0,de.rki.coronawarnapp.ui.main.MainActivity] 08-21 13:22:57.984 4226 4249 I am_activity_launch_time: [0,29948251,de.rki.coronawarnapp/.ui.main.MainActivity,567,567] 08-21 13:22:58.345 4226 4238 I am_destroy_activity: [0,160062039,25836,de.rki.coronawarnapp/.ui.LauncherActivity,finish-imm] 08-21 13:22:58.600 4226 4865 I am_destroy_service: [0,62695349,5331] 08-21 13:22:58.943 4226 6000 I am_crash: [22525,0,de.rki.coronawarnapp,684211780,net.sqlcipher.database.SQLiteException,file is not a database: , while compiling: select count(*) from sqlite_master;,SQLiteCompiledSql.java,-2] 08-21 13:22:58.944 4226 6000 I am_finish_activity: [0,29948251,25836,de.rki.coronawarnapp/.ui.main.MainActivity,force-crash] 08-21 13:22:58.946 4226 6000 I am_home_stack_moved: [0,1,0,0,finishActivity adjustFocus setFocusedActivity] 08-21 13:22:58.946 4226 6000 I wm_task_moved: [25815,1,0] 08-21 13:22:58.952 4226 6000 I am_focused_activity: [0,com.huawei.android.launcher/.Launcher] 08-21 13:22:58.982 4226 6000 I am_pause_activity: [0,29948251,de.rki.coronawarnapp/.ui.main.MainActivity] 08-21 13:22:59.000 4226 4878 I am_create_service: [0,12941628,.DropBoxEntryAddedService,10008,5331] 08-21 13:22:59.015 4226 4237 I am_create_service: [0,258807493,.GmsIntentOperationService,10008,6574] 08-21 13:22:59.046 4226 6001 I am_create_service: [0,168750376,.ReportingAndroidService,10008,5331] 08-21 13:22:59.052 4226 4238 I am_destroy_service: [0,12941628,5331] 08-21 13:22:59.202 4226 6271 I am_proc_died: [0,22525,de.rki.coronawarnapp]

So tried new installation, allow power save disable, but then Ursache 9002 timeout, maybe just no internet connection. Tried again worked, toggle activation off/on no error -> DB access seems to work. Let's wait for next day and new status.

[bugreport-2020-08-21-14-40-44.txt] (no error) 08-21 14:39:34.918 5397 6422 W PGApi_client: recv actoionId = 10000, action = com.huawei.pgmng.PGAction@8d2b235 actionId =10000 pkg =de.rki.coronawarnapp extend1 =355 extend2 = flag =3 type =1 08-21 14:39:34.918 5397 6422 W PGMiddleWare: in handleAction method, action = 10000 08-21 14:39:34.919 5397 6422 W PGMiddleWare: in handleAction, invoke client = com.huawei.pgmng.middleware.AudioEffectLowPowerImpl@564af88, action = com.huawei.pgmng.PGAction@8d2b235 actionId =10000 pkg =de.rki.coronawarnapp extend1 =355 extend2 = flag =3 type =1 08-21 14:39:34.932 4908 5126 I HwSystemManager: NotificationGuideService:handle MSG_ACTIVIY_FOREGROUND, uid:10313

vaubaehn commented 4 years ago

Hi @helmutweick , very big kudos! I think, your support will turn out very helpful! (I hope I'm not wrong...)

So, let's sum up a bit.

  1. First crash on Aug 20, around 23:17. Do you remember, what exactly you did before running CWA in foreground? May there be anything associated with that first crash? As I pointed out here -> https://github.com/corona-warn-app/cwa-app-android/issues/1053#issue-683493833 , I believe the phone's LogCollecting-Service (part of Huawei Analytics/Huawei Mobile Services) threw the exception. The root cause for that exception may be somewhere inside CWA, could get a hard job to find out, where exactly. The crash caused CWA to close immediatly, while the encrypted database of CWA was still open. And so, the database became corrupted.

  2. Next day, you restarted CWA. You found yourself in the 'onboarding', but you couldn't complete because of exception 9002: file is not a database was thrown. I guess, the onboarding was a fallback mechanism. CWA 'discovered', that parts of CWA's data were not valid anymore. It tried to reinitialize, but it failed. Reason could be here -> #1037

  3. After that, you tried a new installation. This completely wipes all of CWA's data, allowing the new encrypted database to be initialized. Everything will be fine, until 1. -> First crash...

[Edit: 4. Yes, the 9002: timeout points out to network connection. Did you restart your phone directly before? Or enabled WiFi just a second before you opened CWA?]

The root cause for all these crashes might turn out to be a hard to detect flaw somewhere in CWA's code. But to try to mitigate, you could try steps suggested here -> https://github.com/corona-warn-app/cwa-app-android/issues/1053#issue-683493833 , where as the first step is to disable

If you would like to support some more, there is one thing you could do: Unfortunately Huawei's logging service hides some logcat entries, according to here -> https://stackoverflow.com/questions/42691076/logcat-not-showing-errors-from-my-huawei-p9-phone You might try the given dial code from above website, to enhance the log files, and maybe post them (with Huawei Mobile Services enabled then!) I assume this to be extremely helpful.

I don't know, @d4rken , is this interesting for you and you may like to have a deeper look into it?

marcelm commented 4 years ago

@marcelm do you remember which type of 9002 it was (there are several)?

It is "file is not a database". I don’t have access to the phone anymore, so the only thing I can say now is that the error (the app closing immediately after start without any message) still occurs every time the app is opened.

Thanks for the prompt replies in any case!

d4rken commented 4 years ago

Thanks for all the reports and logs! Thanks for the coordination efforts @vaubaehn.

I've got my hands on a P8 lite and so far managed to reproduce the issue ONCE, after a reboot. The problem is that it seems to happen less when I'm watching (debugger attached, or logging). The P8 lite isn't very powerful and thus apps not in foreground are prone to be killed by the system to regain resources. Monitoring the app makes it less likely to be killed, which in turn makes the issue more difficult to reproduce, if it is related to process death.

  1. First crash on Aug 20, around 23:17. Do you remember, what exactly you did before running CWA in foreground? May there be anything associated with that first crash? As I pointed out here -> #1053 (comment) , I believe the phone's LogCollecting-Service (part of Huawei Analytics/Huawei Mobile Services) threw the exception. The root cause for that exception may be somewhere inside CWA, could get a hard job to find out, where exactly. The crash caused CWA to close immediatly, while the encrypted database of CWA was still open. And so, the database became corrupted.

The NullPointerException from the Huawei logging service is probably not the cause. I managed to produce the NullPointerException a few times without causing the database issue. I'm not so sure that the database is actually corrupted. The error msg will show if the database can't be decrypted. A corrupted database can't be decrypted, but if we don't have the right key, we also can't decrypt it. #1037 was an unrelated issue with the same symptons. A test method reset the password and used a new one on the old database.

@jakobmoellersap was already suspicious of that but it's tough to confirm anything with this being so rare to reproduce under observation.

Other reports here and here also show stacktraces from the encrypted shared preferences.

  1. Next day, you restarted CWA. You found yourself in the 'onboarding', but you couldn't complete because of exception 9002: file is not a database was thrown. I guess, the onboarding was a fallback mechanism. CWA 'discovered', that parts of CWA's data were not valid anymore. It tried to reinitialize, but it failed. Reason could be here -> #1037

The issue being related to the encrypted SharedPreferences would explain this behavior. That on-boarding is completed, is saved in the same preferences. If nothing is saved yet (or the saved preferences are inaccessible), onboarding will be shown.

[Edit: 4. Yes, the 9002: timeout points out to network connection. Did you restart your phone directly before? Or enabled WiFi just a second before you opened CWA?]

There are multiple errors that lead to code 9002, AFAICT the 9002 timeout is unrelated to this database issue. Due to the P8 lite being "a bit" slow, it also more likely for a 9002: timeout to happen :smile:, also looking into that. I would surprise me if the database issue can be triggered by changing network states.

mmaeusezahl commented 4 years ago

Hi everyone,

although the bug described in this issue is not exactly what I'm encountering I'm still reporting it here since it seems to be very much related, but I'm not running a Huawei device for once.

Setup: OnePlus 6, Magisk rooted

What happend: App ran fine ever since its release until I installed the latest OTA update (to 10.3.5) yesterday. The first time (and ever since) I started the CWA afterwards, the app gets stuck after the "onboarding" stage with the "9002: file is not a database..." error message (of course I had already done the onboarding before).

I'm somewhat confident that reinstalling will be a permanent solution in my case (since I'm not running a Huawei and presume that the issue relates to the system update). But I would be happy to supply logfiles captured while the app crashes in case anyone is interested?

[And some other idea that came to my mind: As I said, I'm running a Magisk rooted phone. To not loose the root during the OTA, I followed the following procedure:

  1. Uninstall all Magisk modules
  2. Remove Magisk from the system images (without reboot)
  3. Install the system update (to the B partition slot, still no reboot)
  4. Patch the waiting update by using the "Magisk A/B retention script"
  5. reboot

I can very much imagine that some crucial encryption data is lost in this A/B switching process?! ^^ On the other hand, Huawei is not using the A/B partition afaik.]

Neuwessi11 commented 4 years ago

Könnte mir bitte mal jemand eine deutsche Zusammenfassung dieses Threads geben? Dieser Fehler ist in meiner Verwandtschaft aufgetaucht und ich würde gerne wissen, ob man nun etwas dagegen tun kann oder nicht.

Grundsätzlich muss ich sagen, dass mir nicht ganz klar ist, warum hier so wenig deutsch geschrieben wird. Immerhin handelt es sich ja um eine App ausschließlich für Deutschland, soweit ich informiert bin.

thomasaugsten commented 4 years ago

@Neuwessi11 Der Fehler tritt häufig auf alten und langsame Geräten auf bei denen es zu speichern in der App Datenbank zu Fehlern kommt. Diese Fehler oder der Verlust der Entschlüsselungsdaten führt dazu das man nicht mehr auf die App zugreifen kann. Wir arbeiten daran in den nächsten Versionen dieses Probleme abzufangen.

Zur Zeit lässt es sich nur lösen in dem man die Daten der App löscht oder sie neuinstalliert Einstellungen -> Apps -> CWA-> Speicher-> Daten löschen

Wir wissen leider nicht immer ob alle Beteiligten deutsch verstehen, deswegen hat man sich auf englisch geeinigt. Aber wir antworten auch auf deutsch.

Neuwessi11 commented 4 years ago

@thomasaugsten Ganz herzlichen Dank!!! Gehen denn beim Datenlöschen oder der Neuinstallation nicht die gesamten Kontaktschlüssel verloren? Und ist die App nicht dadurch im Moment für von diesem Fehler Betroffene mehr oder weniger nutzlos, wenn der Fehler innerhalb von ca. 14 Tagen auftritt?

Was ich trotzdem noch nicht verstehe: Angesichts der Tatsache, dass es sich um eine von zwei deutschen Unternehmen für Deutschland entwickelte App handelt, sollten doch eigentlich mehr Leute Deutsch als Englisch verstehen (Beispiele: Meine gesamte Verwandtschaft und ich verstehen Deutsch um Klassen besser als Englisch). Wo ist mein Denkfehler?

daimpi commented 4 years ago

@Neuwessi11 Es gibt wahrscheinlich einige EntwicklerInnen und NutzerInnen der CWA die kein Deutsch verstehen. Englisch ist einfach eine viel universellere Sprache, gerade auch wenn es um Austausch mit Google/Apple geht (die das ENF bereitstellen) oder den Kontakt mit Teams in anderen Ländern die ENF basierte Apps bauen. Deswegen bin ich ehrlich gesagt froh, dass Englisch die Standardsprache ist.

thomasaugsten commented 4 years ago

@Neuwessi11 Die Kontaktschlüssel werden vom Betriebssystem gespeichert und unser Stand ist, das nach dem Löschen der letzten Covid-19 App die Schlüssel in Android auch gelöscht werden. Wenn man über Einstellungen -> Apps -> CWA-> Speicher-> Daten löschen geht kann man die App zurücksetzen ohne die App zu löschen.

Ihre Annahme ist richtig das in unseren Unternehmen mehr Leute Deutsch verstehen. Aber nicht alle Nutzer der App verstehen Deutsch und gerade dieses Thema ist auch für Google und andere Länder die eine Corona-Warn-App bereitstellen relevant.

UPDATE: Wann gelöscht wird präzisiert.

daimpi commented 4 years ago

@thomasaugsten

[…] unser Stand ist, das nach dem Löschen der App die Schlüssel im Betriebssystem erhalten bleiben.

I thought the keys are only preserved under iOS, but not under Android when uninstalling the app:

On Android devices, the log is deleted if Corona-Warn-App was the last app using the framework on the device. If you have SwissCovid or Immuni installed in addition to the Corona-Warn-App, the log will only be removed if these additional apps are de-installed or you manually trigger deletion of the exposure log.

https://www.coronawarn.app/en/faq/#delete_random_ids

thomasaugsten commented 4 years ago

This is correct for the first versions of the Google ENF. I will double check with google what's the current status of this in the latest version. But data deletion via Einstellungen->Apps should not delete the keys on all Google ENF Versions.

daimpi commented 4 years ago

That would actually be great if Google changed this 😊

vaubaehn commented 4 years ago

Hi @thomasaugsten ,

This is correct for the first versions of the Google ENF. I will double check with google what's the current status of this in the latest version. But data deletion via Einstellungen->Apps should not delete the keys on all Google ENF Versions.

thanks for being into it. There are some hints that there might have been changes with the latest ENF versions, see https://github.com/corona-warn-app/cwa-app-android/issues/1053#issuecomment-678381760 and following.

Have a good day!

vaubaehn commented 4 years ago

Hi @d4rken , thanks so much for your reply and your time, it's very nice to hear the current position in this issue from an expert's perspective!

I've got my hands on a P8 lite

Bingo. :)

and so far managed to reproduce the issue ONCE, after a reboot. The problem is that it seems to happen less when I'm watching (debugger attached, or logging). The P8 lite isn't very powerful and thus apps not in foreground are prone to be killed by the system to regain resources. Monitoring the app makes it less likely to be killed, which in turn makes the issue more difficult to reproduce, if it is related to process death.

That's bad, actually (and did you author the German entry for 'race condition' in Wikipedia? Nearly same words in second paragraph...)

As I pointed out here -> #1053 (comment) , I believe the phone's LogCollecting-Service (part of Huawei Analytics/Huawei Mobile Services) threw the exception. The root cause for that exception may be somewhere inside CWA, could get a hard job to find out, where exactly. The crash caused CWA to close immediatly, while the encrypted database of CWA was still open. And so, the database became corrupted.

The NullPointerException from the Huawei logging service is probably not the cause. I managed to produce the NullPointerException a few times without causing the database issue. I'm not so sure that the database is actually corrupted.

To be honest, I'm also not 100% sure, if (all) exceptions are related to database corruption. Crucial point is, if there can be any condition where the database is not closed / still written to, when any crash occurs. If in the debug- / test-setting the task has a higher priority / more ressources, then it might get difficult, to hit that exact point.

The error msg will show if the database can't be decrypted. A corrupted database can't be decrypted, but if we don't have the right key, we also can't decrypt it. #1037 was an unrelated issue with the same symptons. A test method reset the password and used a new one on the old database.

If I understood it right, the current fallback mechanism of data storage for the productive app is (only) to set a new dbPassword, if data got corrupted, is that correct? That would explain any security-related exception at the point of trying to read from database, when the database file still exists in any state.

@jakobmoellersap was already suspicious of that but it's tough to confirm anything with this being so rare to reproduce under observation.

Yes, indeed. @jakobmoellersap considered two things: Either an "incompatible crypto provider under the hood of SecureRandom which corrupts the password generation". But I wonder, if in that case the exception then would occur regularly, already from the first run of CWA. But users report, that CWA often runs flawless for them a couple of days, if not even weeks, before a crash occurs. Or, the second consideration of @jakobmoellersap , "another suspect (hopefully not applicable) would be that the devices actually try to look into the database files or the encrypted shared preferences and alter them somehow, making the read corrupt as well." But would something like this not be very risky for Huawei, if some day a sophisticated developer or security engineer finds out about it? In that case Huawei would risk to get banned from whole markets with their devices... Uh-oh, ehm, well.. So, that's maybe possible. But there are some 'but's:

Other reports here and here also show stacktraces from the encrypted shared preferences.

Yes, we have two different types of exceptions.

  1. Next day, you restarted CWA. You found yourself in the 'onboarding', but you couldn't complete because of exception 9002: file is not a database was thrown. I guess, the onboarding was a fallback mechanism. CWA 'discovered', that parts of CWA's data were not valid anymore. It tried to reinitialize, but it failed. Reason could be here -> #1037

The issue being related to the encrypted SharedPreferences would explain this behavior. That on-boarding is completed, is saved in the same preferences. If nothing is saved yet (or the saved preferences are inaccessible), onboarding will be shown.

If I strictly follow my own assumptions from above: EncryptedSharedPreferences are corrupted. -> CWA starts onboarding. (Does it erase any data before or during onboarding, to be sure, to initialize a clean set-up of ESP and DB? Can corrupt files on filesystem level easily be deleted or overwritten by CWA?) -> CWA initializes a new db + password. -> it tries to write to that db -> 9002: file is not a database. My pathway of assumptions can only be correct, if

[Edit: 4. Yes, the 9002: timeout points out to network connection. Did you restart your phone directly before? Or enabled WiFi just a second before you opened CWA?]

There are multiple errors that lead to code 9002, AFAICT the 9002 timeout is unrelated to this database issue. Due to the P8 lite being "a bit" slow, it also more likely for a 9002: timeout to happen smile, also looking into that. I would surprise me if the database issue can be triggered by changing network states.

Yes, 9002: timeout was also one of my 'babies' :) #998 I completely agree with you, that it's unrelated. I just mentioned it to complete commenting on all of @helmutweick 's observations.

I think, one way to workaround this whole issue for now, is to backup ESP and DB in local storage, let CWA catch related exceptions, and in that case restore both files from backup (works only, if Android masterkey is not altered). That's not elegant at all. But maybe provides a feasible way, until the root causes are found and a more sophisticated solution is found.

Thanks again for your time, and would be happy to hear from you again!

cwaUser commented 4 years ago

Same error with different call stack at Huawei P8, Android 6.0, cwa 1.2.1:

Screenshot_2020-08-26-06-54-17 Screenshot_2020-08-26-07-02-18 Screenshot_2020-08-26-07-03-00 Screenshot_2020-08-26-07-03-17 Screenshot_2020-08-26-07-03-35 Screenshot_2020-08-26-07-05-09

d4rken commented 4 years ago

A little status update :coffee:.

I've not managed to produce the error again :frowning_face: . I will carve out more time soon to spend some hours on manually stepping through the code to try to force a reproduction by interfering with the EncryptedSharedPreferences.

I'm also evaluating with our team whether we can make something like community test-builds available to include you closer in debugging process, especially in cases like this elusive bug. There are "alpha" updates available for the CWAs androidx.security dependency where the changelog reads like it could help us with this issue. Because it's an alpha update, I don't want to just upgrade it without being able to confirm it as fix as it could potentially break other things.

vaubaehn commented 4 years ago

Hi @d4rken , thanks a lot for providing an update here!

I've not managed to produce the error again :frowning_face: . I will carve out more time soon to spend some hours on manually stepping through the code to try to force a reproduction by interfering with the EncryptedSharedPreferences.

It's frustrating, but thanks for being on it, and we can be sure that issue is in good hands here.

I'm also evaluating with our team whether we can make something like community test-builds available to include you closer in debugging process, especially in cases like this elusive bug.

I think it's a good idea, and I bet there are at least some volunteers that would love to engage in that way. Personally, I'm lacking a test-device... If someone wants to donate? P8 lite preferably ;)

There are "alpha" updates available for the CWAs androidx.security dependency where the changelog reads like it could help us with this issue. Because it's an alpha update, I don't want to just upgrade it without being able to confirm it as fix as it could potentially break other things.

Yes, the change logs sound somehow promising, and your caution is certainly in order here.

As a cross-reference, in https://github.com/corona-warn-app/cwa-app-android/issues/579#issuecomment-678909159 @hannesa2 brought up implementation 'net.zetetic:android-database-sqlcipher:4.4.0' into the game. He seems to be specifically experienced with sqlcipher, and maybe he likes to provide more information here, or give some concrete suggestions/recommendations on how to continue EncryptedSharedPreferences and encrypted sql lite database without running into crashes mainly on Huawei/Honor devices (and more rarely others)?

Annie-G commented 4 years ago

Running the App on Lenovo Moto Z (XT1650-03) Build OPL27.76-71-2-3, Android 8.0.0 Used to run ok for several months. Now crashes since I switched on my phone this morning. Getting no error notifications except crash note. Not able to open the app. Had to delete App data. Now the app starts again, but no more risk status because the data is lost. RKI told me in their reply to my comment in playstore that this (cause 9002) might be the problem ans I should add my type of phone here if not yet mentioned here.

Nochmal auf deutsch: Ich habe ein Lenovo Moto Z (XT1650-03) Build OPL27.76-71-2-3, Android 8.0.0 App lief mehrere Monate lang. Seit heute, nach Einschalten des Geräts, kommen permanent Absturz-Meldungen "Corona-Warn-App wurde beendet". Andere Fehlermeldungen kriege ich nicht, nur das, auch wenn ich die App selbst starten will. Hab eine Bewertung in den playstore geschrieben, RKI antwortete mit Verweis auf diesen Fehler (cause 9002). Ich bin nicht sicher, ob es das ist. Aber ich habe hier gelesen, das Löschen der App-Daten behebt erstmal diesen Fehler. Das hat bei mir funktioniert, nur habe ich jetzt keinen Risikostatus mehr mangels Daten.

hannesa2 commented 4 years ago

As a cross-reference, in #579 (comment) @hannesa2 brought up implementation 'net.zetetic:android-database-sqlcipher:4.4.0' into the game. He seems to be specifically experienced with sqlcipher, and maybe he likes to provide more information here, or give some concrete suggestions/recommendations on how to continue EncryptedSharedPreferences and encrypted sql lite database without running into crashes mainly on Huawei/Honor devices (and more rarely others)?

We spent month with unexplained corruptions which we observed via Crashlytics. Approximately 1,5 % of all users where involved (of 5 millions). Most of them where Samsung devices, but there were no exact rule. After a half year we eliminate this sqlite secure library (the maintainer are snooty and gave stupid advises) and stored only one single token in keystore.

vaubaehn commented 4 years ago

Hi @hannesa2 , thank you very much for your reply. The developers, reading along here, are now able to take it into account.

vaubaehn commented 4 years ago

Hi @thomasaugsten ,

@Neuwessi11 Die Kontaktschlüssel werden vom Betriebssystem gespeichert und unser Stand ist, das nach dem Löschen der letzten Covid-19 App die Schlüssel in Android auch gelöscht werden. Wenn man über Einstellungen -> Apps -> CWA-> Speicher-> Daten löschen geht kann man die App zurücksetzen ohne die App zu löschen. [...}

[...}. I will double check with google what's the current status of this in the latest version. But data deletion via Einstellungen->Apps should not delete the keys on all Google ENF Versions.

do you already have a feedback from Google? It would be very helpful to support users coming in here, with apps crashing on start. Will collected RPIs/TEKs associated with CWA be deleted, when a) deleting data and cache via Android > settings > apps > CWA > storage ? b) uninstalling CWA, if it's the only contract tracing app on the device? c) uninstalling CWA, when there are other contract tracing apps (e. g., SwissCovid) are staying on the device?

As for now it seems, that at least the exposure logs inside ENF are deleted in either case a), b), c). Knowing what happens to RPIs/TEKs is quite crucial.

Thank you very much in advance for your reply!

vaubaehn commented 4 years ago

@thomasaugsten , one additional question: a registered QR-code would also be deleted by deleting app data (gone forever, no re-registration possible), right?

thomasaugsten commented 4 years ago

This is correct the QR cannot added again to the CWA.

vaubaehn commented 4 years ago

dear @thomasaugsten , thanks a lot for replying to my additional question. Could you also give an update to my original question? Thank you!

Hi @thomasaugsten ,

@Neuwessi11 Die Kontaktschlüssel werden vom Betriebssystem gespeichert und unser Stand ist, das nach dem Löschen der letzten Covid-19 App die Schlüssel in Android auch gelöscht werden. Wenn man über Einstellungen -> Apps -> CWA-> Speicher-> Daten löschen geht kann man die App zurücksetzen ohne die App zu löschen. [...}

[...}. I will double check with google what's the current status of this in the latest version. But data deletion via Einstellungen->Apps should not delete the keys on all Google ENF Versions.

do you already have a feedback from Google? It would be very helpful to support users coming in here, with apps crashing on start. Will collected RPIs/TEKs associated with CWA be deleted, when a) deleting data and cache via Android > settings > apps > CWA > storage ? b) uninstalling CWA, if it's the only contract tracing app on the device? c) uninstalling CWA, when there are other contract tracing apps (e. g., SwissCovid) are staying on the device?

As for now it seems, that at least the exposure logs inside ENF are deleted in either case a), b), c). Knowing what happens to RPIs/TEKs is quite crucial.

Thank you very much in advance for your reply!

mh- commented 4 years ago

a) deleting data and cache via Android > settings > apps > CWA > storage ? b) uninstalling CWA, if it's the only contract tracing app on the device? c) uninstalling CWA, when there are other contract tracing apps (e. g., SwissCovid) are staying on the device?

In my experience, only b) will delete collected RPIs/TEKs.

However, I think if you do a) or c), and run CWA again after that, CWA will assume that it was freshly installed and will not download keys for the previous days -> only very little matching will happen. At least it was like this a few weeks ago. You had to manually change the device's clock for the first CWA start to mitigate this.

Annie-G commented 4 years ago

a) deleting data and cache via Android > settings > apps > CWA > storage ? b) uninstalling CWA, if it's the only contract tracing app on the device? c) uninstalling CWA, when there are other contract tracing apps (e. g., SwissCovid) are staying on the device?

In my experience, only b) will delete collected RPIs/TEKs.

However, I think if you do a) or c), and run CWA again after that, CWA will assume that it was freshly installed and will not download keys for the previous days -> only very little matching will happen. At least it was like this a few weeks ago. You had to manually change the device's clock for the first CWA start to mitigate this.

Interesting! Though I think I'm too stupid to understand this... So I'd do a) and afterwards, before I start the app again, manually change the phones clock. To what time?

vaubaehn commented 4 years ago

@Annie-G If I understood it correctly:

Annie-G commented 4 years ago

@Annie-G If I understood it correctly:

* You would do a).

* Then you would set your systems date 14 days back to the past.

* Then you would open CWA, and wait until you have any risk status.

* Then you would close CWA.

* Then you would set your systems date to the current date.

* Then you would open CWA again, and hopefully, the diagnosis keys for the past 14 days are downloaded and matched now.

Yassssss thank you, that worked.

vaubaehn commented 4 years ago

@Annie-G great! Could you do us a favor? Could you please look to Android > settings > google > Covid-19-Benachrichtigungen > Überprüfungen auf mögliche Begegnungen Of how many days do you find entries here?

Annie-G commented 4 years ago

@Annie-G great! Could you do us a favor? Could you please look to Android > settings > google > Covid-19-Benachrichtigungen > Überprüfungen auf mögliche Begegnungen Of how many days do you find entries here?

Done. It says 28 for 14 days. Well that's not much. Before the app crashed and I deleted app data, there were 214 entries for the last two weeks.

vaubaehn commented 4 years ago

@Annie-G Again, thanks for your feedback. This is, because for two days (today-14 days, today) the 14-day-packages to check for exposure encounters have been downloaded and checked. Anyhow, the relevant last 14 days should have been checked for encounters now. The expected side effect should be, that CWA is now not showing an undefined risk state (nicht "Risiko konnte nicht ermittelt werden"), but a (hopefully) green state. Is that correct? And "days active" should most likely be 14 of 14 days active?