search

corona-warn-app / cwa-app-android

Native Android app using the Apple/Google exposure notification API. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
2.44k stars 495 forks source link

Error when communicating with Google API(10): Unable to validate key file signature. #737

Closed stefan-sys closed 4 years ago

stefan-sys commented 4 years ago

Auf Gerät mit Android 6.0

Scan 24.06.2020.pdf

Internal Tracking ID: EXPOSUREAPP-1714

Tj20 commented 4 years ago

Quick question: My app seems to have stopped working at all now, because of the error mentioned here. It says: "Risiko-Ermittlung nicht möglich, Ihre Risiko-Ermittlung konnte seit mehr als 24 Stunden nicht aktualisiert werden"

I'm sure that this bug will get fixed eventually, so I'll just wait and see. I'm just wondering: Should I keep the app running for now, or does it not do anything at all anymore in this state, so there's no benefit of keeping the app running and Bluetooth on at the moment?

jakobmoellerdev commented 4 years ago

Please keep the App running. As mentioned above, your key generation and the possibility to warn others are not affected.

Tj20 commented 4 years ago

Thanks, I see!

kbobrowski commented 4 years ago

@jakobmoellersap right, this is definitely not a good final solution. Agree that this should never be the behavior of the API.

The rationale for this PR is based on:

jakobmoellerdev commented 4 years ago

Totally agree @kbobrowski.

So I will follow-up as soon as I have a decision on our side on how we want to proceed but I want to make a few things clear:

If we do not find a solution fast enough in the next few days we will go ahead and workaround (possibly with a retry solution or something that combines multiple approaches).

christianneu commented 4 years ago

Here is another device with the problem, which worked 9 days fine:

App Version: 1.0.4 Android Version: Android 6.0 Device Model and Manufacturer: Honor 7 Latest OS Update:? Latest Update of CWA: 27.06.20 Installation Date of CWA: 27.06.20 Date/Time of the error: 27.06.20 / 10:00 am

image

jakobmoellerdev commented 4 years ago

The error will appear sporadically and is not persistent from what we know so far. Please only post stack trace information or pictures in case you do not find the same error in this thread. Thanks everyone!

kbobrowski commented 4 years ago

@jakobmoellersap it does not seem to take quota if there is an error, at least not when 39506: No such file or directory is raised, tried calling it 32 times, first time with real path, 30 times with fake path and last time with real path again:

call number 1: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/81a2d7a0-6d15-31eb-b89b-8b648347aaaa.zip])
call number 2: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/does-not-exist.zip])
call number 3: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/does-not-exist.zip])
call number 4: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/does-not-exist.zip])
call number 5: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/does-not-exist.zip])
on failure listener: com.google.android.gms.common.api.ApiException: 39506: No such file or directory
on failure listener: com.google.android.gms.common.api.ApiException: 39506: No such file or directory
on failure listener: com.google.android.gms.common.api.ApiException: 39506: No such file or directory
(...)
on success listener: null
(...)
call number 31: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/does-not-exist.zip])
on failure listener: com.google.android.gms.common.api.ApiException: 39506: No such file or directory
call number 32: provideDiagnosisKeys(keys=[/data/local/tmp/key-export/81a2d7a0-6d15-31eb-b89b-8b648347aaaa.zip])
on failure listener: com.google.android.gms.common.api.ApiException: 39506: No such file or directory
on success listener: null
jakobmoellerdev commented 4 years ago

When you called the method, did you use a whitelisted device? Then, the limit would be set way higher

kbobrowski commented 4 years ago

Sorry correction - the quota is not taken if the file does not exist, but it is taken in case of error 10 - after exactly 20 calls there is 39508 error, regardless of whether these calls were successful or throwing error 10. I'm not whitelisted

on success listener: null
on success listener: null
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on success listener: null
on success listener: null
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on success listener: null
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on success listener: null
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on success listener: null
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on success listener: null
on success listener: null
on success listener: null
on success listener: null
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on failure listener: com.google.android.gms.common.api.ApiException: 10: Unable to validate key file signature: Pipe is closed
on failure listener: com.google.android.gms.common.api.ApiException: 39508: 
on failure listener: com.google.android.gms.common.api.ApiException: 39508: 
on failure listener: com.google.android.gms.common.api.ApiException: 39508:
jakobmoellerdev commented 4 years ago

Got it. This would actually be great for us, this means that your retry solution is a valid workaround. I will check this again with a colleague to make sure we don't miss anything and if everything goes okay we will have a hotfix while working on a final solution in parallel. Since this is such a sensitive part of the application we do not want to introduce more error potential with this.

Edit: Nevermind, read that post too fast. My bad. In case we influence the rate limiting with this error, any retry is dangerous and we need to avoid this.

kbobrowski commented 4 years ago

Unfortunately retry solution may result in exceeded quota, since error 10 is taking the quota. But if it happens rarely and requires just couple of retries, then it should fit in 20 calls even with 14 diagnosis keys. I'm doing some further tests

kbobrowski commented 4 years ago

@jakobmoellersap I can no longer reproduce this error if I put 10 second delay between calls to provideDiagnosisKeys

More detailed statistic of number of exceptions vs delay will follow

helmutweick commented 4 years ago

Here is a bug report for the API(39508) error, which occurs some time after error 10. So far (since 12h) I could not get rid of it (phone on/off, reinstall app, toggle blue I do not even see the error 10 anymore, I am blocked by too many bluetooth attempts.

bugreport-2020-06-27-15-03-18.txt tooth).

jakobmoellerdev commented 4 years ago

39508 will be gone after one day since this is the Rate Limiting of the API which resets after 1 day.

kbobrowski commented 4 years ago

If it would take long to remove this error by rolling out Play Services update then introducing 10 seconds delay seems to be solving this immediatelly

errors_vs_delay

Dakror commented 4 years ago

Im experiencing the same issue. My initial and possibly dumb reaction was reinstalling the app. Didnt think about the data getting wiped. Is the app able to resume my key chain(from some internal or Google based initial keys possibly) or is my chain of keys broken?

tkowark commented 4 years ago

Im experiencing the same issue. My initial and possibly dumb reaction was reinstalling the app. Didnt think about the data getting wiped. Is the app able to resume my key chain(from some internal or Google based initial keys possibly) or is my chain of keys broken?

Your chain of keys is not broken. These are not stored by the app and deleting the app will not remove the keys. To do that, you have to explicitly remove them via the Settings for the Covid-19 Exposure Protocol.

stefanbschneider commented 4 years ago

Same issue for me on Honor 6: https://github.com/corona-warn-app/cwa-app-android/issues/602#issuecomment-650734091

wende60 commented 4 years ago

Same issue with HTC One A9s, Android 6.0

oschmidtmann commented 4 years ago

Same issue with Honor 5x (KIW-L21), Android 6.0.1 after a few days without any error messages. I kept trying to get to the bottom of it (blaming my Huawei), I finally cleared the Google caches, updated the Google services and reinstalled the app. My data has been wiped: I am back to day 1 – which is not really an issue for me. After initial error messages, is seems to work fine, again. Thanks for looking into it.

Dakror commented 4 years ago

Im experiencing the same issue. My initial and possibly dumb reaction was reinstalling the app. Didnt think about the data getting wiped. Is the app able to resume my key chain(from some internal or Google based initial keys possibly) or is my chain of keys broken?

Your chain of keys is not broken. These are not stored by the app and deleting the app will not remove the keys. To do that, you have to explicitly remove them via the Settings for the Covid-19 Exposure Protocol.

Thank you for clarifying that! Glad to see you built that in!

sinaler commented 4 years ago

Same issue on Huawei ALE-L21 as well.

helga-engels commented 4 years ago

Same issue on Huawei p9 lite Android 6.0 (API 39508 bzw. API 10)

Update: Yes, Error 39508 doesn't appear any longer.

Some stupid Question: There are so many HUAWEI-Smartphones reporting Google-API-Error 10, so mine. I don't have Google Mobile Services (GMS) but Huawei Mobile Services (HMS) on my smartphone. Is there probablay any relation?

Martin-Luft commented 4 years ago

Same here Honor 7 Premium 32 GB (Android 6). The problem popped up yesterday morning but after a few hours it disappeared.

Since 4 days the error does not disappear and the app shows me the warning, that the "Risiko-Ermittlung" did not work since 25th of June. I reinstalled the app but nothing changed. In other words, the app is not useable at all on my smartphone :(

AHaumer commented 4 years ago

Seems to be the same here: CWA 1.0.4 Acer T04, Android 6.0, Kernel 3.18.19, not rooted, mobile data connection switched off, WLan and Bluetooth switched on without "Risiko-Ermittlung" (since 2020-06-26) useless

WHermann-62 commented 4 years ago

The same bug on a Huawei P8 (GRA-L09) / Android 6.0 / EMUI 4.0.3. The app ran without errors for about 10 days; on 27.06. this error message appeared. After restarting the P8, the app was in the same state as when reinstalled. After that, error 9002 came up and I couldn't read much because the app crashed completely and was closed. After deleting the app and reinstalling it I got "Error communicating with Google API(10) ..." again.

HerbyR56 commented 4 years ago

Same Bug here on my and my wifes phone, both are Huawei P8 Lite, looks like it was coming after update to 1.04. Error is permanent, not sporatic. Android 6.0

PetDD commented 4 years ago

CWA lief problemlos über 9 Tage bis 25. / 26. Juni. Danach verschiedene Fehlermeldungen z.B. Kommunikation mit Google API 10 unable to validate key file signature../ API 17... / .9002 .file is not a database .../ .Timeout...) Alle Problemlösungsvorschläge durchgearbeitet /aktualisieren/Daten+Cache löschen. Neustart. Ländereinstellung im PlayStore einstellen nicht möglich, da ich keine Zahlungsmittel hinterlegen will. Es stand in den Anleitungen, das der Google Mobile Service Updates durchführt. ABER: Auf meinem Huawei P8 (ALE-L21), sehe ich unter Apps keinen Google Mobile Service. Es gibt nur einen Huawei Mobile Service in meinen Apps. Nach mehreren Versuchen Neuinstallation CWA, dauerhaft sofort bei Start Fehler Ursache 3. Huawei P8 (ALE-L21), Android 6.0, EMUI 4.0.3

kbobrowski commented 4 years ago

I think I have solved the problem - PipedInputStream.read(byte[] buffer) seems to be buggy, and the error happens when EK Export v1 is being validated - read(byte[] buffer) method sometimes reads less than 16 bytes. It seems that PipedInputStream.read() could be used instead.

Detailed description is here: https://github.com/kbobrowski/en-api-exception-10-debug#bug-description

After patching it I can no longer reproduce this error on my device even with very frequent call to provideDiagnosisKeys. But this would of course had to patched by Google and rolled out via Play Services update. As an intermediate solution it seems that if calls are separated by some delay then PipedInputStream.read(byte[] buffer) always behaves properly.

jakobmoellerdev commented 4 years ago

@kbobrowski Thanks for actively contributing. We are forwarding your information to Google and will see what happens. The problem with the delay is that we at max have 140 seconds delay, which would result in us having to change transaction timeouts as well as a really bad UX by default. I am currently thinking of a backoff multiplier solution on failure, but we did not decide on the way forward yet.

jakobmoellerdev commented 4 years ago

@HerbyR56 The Error 10 IS in fact sporadic, however as can be seen from the graphic provided above by @kbobrowski, the error probability can get so high that the error appears statistically in above 80% of all tries when querying with more than 10 packages (which for CWA is the case as we have over 10 day packages)

HerbyR56 commented 4 years ago

If you install the Corona App new, it never gets connected, so it's a permanent Error. We have this now since days, it's no more connecting. It says always "Unknown Risk2: then it says Error 3 (Google API 10), again and again, I'm Developer, so for me it's not sporatic, it's systematic.

HerbyR56 commented 4 years ago

And I got also sometimes The Error from Google API(39508), but this is sporatic

aurisnoctis commented 4 years ago

@HerbyR56 The Google API(39508) error is explained here, https://github.com/corona-warn-app/cwa-app-android/issues/774#issuecomment-650968797, and may follow other errors when the API is called too many times in vain, for instance while producing the Google API(10) error too many times.

helga-engels commented 4 years ago

Now receiving error “timeout“ https://drive.google.com/file/d/1BZ-H3Kr6AllXBGtSYmmj99D5hADFRcMw/view?usp=drivesdk

jakobmoellerdev commented 4 years ago

Hey everyone,

First of all, sorry for us being silent about this but there was quite some investigation going on. The issue is identified and actively being worked on in a joint effort and I will write here once a fix is ready. We are terribly sorry for the issues and trust concerns this causes and caused up until now and we will try to mitigate as fast as we responsibly can. Also special shoutout to @kbobrowski for his amazing work on reverse engineering GMS that gave us the heads-up into the right direction.

The Error will also resolve most rate-limiting problems (39508) once fixed, as the pipeline breakdown also caused the limit to be reached on retry.

Best Regards on behalf of the Android Team, Jakob

jakobmoellerdev commented 4 years ago

@helga-engels you had a regular HTTP Timeout, please check your internet connection.

HerbyR56 commented 4 years ago

Thank you @jakobmoellersap for your effort and doing this fantastic work and to @kbobrowski who has identified the problem. @helga-engels: If it's only a timeout I would expect an Error Message like "Timeout, redo it after some time" or so and not this Error, which looks like "General Error". We must take care, that people trust this App. We are on the way to loose the trust.

Maha781 commented 4 years ago

Hi I had a different problem (API(39508) and asked for technical support on the phone. They recommended to uninstall and reinstall which I did. Now I have the problem mentioned above error communicating with API(10). I am using Samsung Galaxy J7.

morber13 commented 4 years ago

Hi I had a different problem (API(39508) and asked for technical support on the phone. They recommended to uninstall and reinstall which I did. Now I have the problem mentioned above error communicating with API(10).

I followed this hint. All data was lost and after one day I get error communicating with API(10).

morber13 commented 4 years ago

This is not a good reputation for this app, I think. "Normal" users will refuse installing it when they read about the problems. See https://www.zeit.de/digital/2020-06/corona-warn-app-bugs-hinweise

tkowark commented 4 years ago

I followed this hint. All data was lost and after one day I get error communicating with API(10).

Just a quick clarification: All APP data was lost, that means the number of active days and your current risk status. The Rolling Proximity Identifiers stored in the exposure log are not deleted when you reinstall that app, but you'd have to do that manually through the covid-19 exposure logging settings. As described here we will again clarify this w/ Google.

helga-engels commented 4 years ago

There is no software without bugs. But this is not the platform for complaining this, but for contributing to the solution with informations, error logs, ideas etc.☺

Maha781 commented 4 years ago

I just spoke to the technical support. They explained more about this API(10) as well as API(39508). It is not a problem with the app but with the communication system. Google is informed, as I was told, and there shall be a solution from google's side in the next few days.

aurisnoctis commented 4 years ago

Update to https://github.com/corona-warn-app/cwa-app-android/issues/737#issuecomment-650144069: Google API(10) error occurred again today, alternating with the Google (20) error until the Google API(39508) timeout error appears, documented here: https://github.com/corona-warn-app/cwa-app-android/issues/788#issuecomment-652871916

Update: API(10) error also occurred on 3 Jul 2020, but without API(20)

helga-engels commented 4 years ago

After deleting complete data (Google Play-Dienste, Corona-Warn) nevertheless permanent communication error API(10).

The focus seems to be on bugs in google play services, but they were not updated shortly before first appearance of error API(10). And the error is still remaining after deleting all data. What about bugs in server software?

crazyduck33 commented 4 years ago

Das ist doch echt einfach nur peinlich. Da wird massig Geld vom Staat in die App versenkt, das ganze dauert Monate wird als "DIE APP" schlechthin beworben von Politik und anderen. Dann installiert man sich die App das ding läuft mal so mal so aber eher gar nicht sondern meldet permanent API 10 fehler.

Hat das eigentlich niemand getestet? Oder hat Google sich entschieden nachdem die App fertig war jetzt mal schnell was an der API zu ändern damit die App nicht mehr sauber funktioniert? Wurde anfangs nicht dafür geworben das die Leute vertrauen haben sollten ? Aber wie soll man einer App vertrauen die in keinster Weise das macht was sie soll?

Ernnsthaft das ding is kein Candy Crush zum rumdaddeln wo Bugs zwar ärgerlich sind aber nich weiter problematisch.

alois31 commented 4 years ago

@jakobmoellersap is there an approximate ETA from Google when they will fix their implementation? Preferably before the exposure keys start expiring.

tkowark commented 4 years ago

Das ist doch echt einfach nur peinlich. Da wird massig Geld vom Staat in die App versenkt, das ganze dauert Monate wird als "DIE APP" schlechthin beworben von Politik und anderen. Dann installiert man sich die App das ding läuft mal so mal so aber eher gar nicht sondern meldet permanent API 10 fehler.

Hat das eigentlich niemand getestet? Oder hat Google sich entschieden nachdem die App fertig war jetzt mal schnell was an der API zu ändern damit die App nicht mehr sauber funktioniert? Wurde anfangs nicht dafür geworben das die Leute vertrauen haben sollten ? Aber wie soll man einer App vertrauen die in keinster Weise das macht was sie soll?

Ernnsthaft das ding is kein Candy Crush zum rumdaddeln wo Bugs zwar ärgerlich sind aber nich weiter problematisch.

Dear @crazyduck33 , even though we understand your anger, we kindly ask you to focus on the issue and not use the comments for unrelated discussions that do not help in getting this issue fixed. If we get more unrelated comments, we will have to lock the conversation and provide updates only when available.

Of course, this app has been tested intensively, and the actual development by SAP and Telekom only started beginning of may. The error occurred only in productive usage and was not present beforehand. Also while we this error occurs on some devices, it does not affect all of them and therefore does not render the app unusable for everybody.

To also answer @alois31 's question: no, we unfortunately do not have an ETA for the fix.