"Risiko-Ermittlung gestoppt" though it had not been turned off

kereng5 commented 3 years ago

Describe the bug

gestoppt_5168 I did not stop the exposure check, and it was active at 2:46 am. So who stopped it and why?

One more small issue: When I click "Aktivieren" to reactivate the exposure check, switch the slider to the right and return to the previous screen by the "back triangle", it still shoes "Risiko-Ermittlung gestoppt", and the slider is left again. It works if I switch the slider to the right, close the CWA and start it new.

Steps to reproduce the issue

I just start the phone in the morning and read "Risiko-Ermittlung gestoppt" in the CWA. It happened today and on the day before yesterday, but not yesterday.

Technical details

Mobile device: Motorola G3
Android version: 6.0.1
CWA version 2.1.2
ENF 18211213000
no "Priorisierte Hintergrundaktivität"

Internal Tracking ID: EXPOSUREAPP-7095

dsarkar commented 3 years ago

@kereng5 Thanks for the report. We getting since version 2.1 several of these reports.

Can you confirm that you have a stable internet connection?
Can you confirm that this issue araised only after updating to CWA version 2.1

Corona-Warn-App Open Source Team

DerVogel2020 commented 3 years ago

This is similar to #2880 .

dsarkar commented 3 years ago

Internal Tracking ID: EXPOSUREAPP-7095

vaubaehn commented 3 years ago

Hi @DerVogel2020 and all, it's not similar to #2880. In my case, a different error occured ("Risiko-Überprüfung fehlgeschlagen" vs. "Risiko-Ermittlung gestoppt"), and contact tracing ("the switch") was still enabled.

I wonder, if this is a problem in ENS which is (correctly) reflected in CWA? (ENS got a hick up, and turned itself off, or something). @kereng5 how is the memory situation of your internal storage? How much free space is in your internal memory?

MikeMcC399 commented 3 years ago

@kereng5

ENF 18211213000

It may be worth checking if there is an update to the Exposure Notifications System for your device, in case it is an issue with this part of the system.

On an Android 8 device I can do: Einstellungen > Apps > Google Play-Dienste tap on App-Details in Store

If there is an update available, it will offer it to you. That may be different on your device / your Android 6.

I'm currently using 21.15.15 for Google Play-Dienste.

In Einstellungen > Google > COVID-19-Benachrichtigungen (scroll down to bottom) this shows as: Version v1.8 (211515000).

dsarkar commented 3 years ago

@kereng5 Could you provide us with a screenshot please:

Einstellungen -> COVID-19-Benachrichtigungen

Thanks.

Corona-Warn-App Open Source Team

kereng5 commented 3 years ago

@dsarkar

* Can you confirm that you have a stable internet connection?
Yes, the phone is in my WLAN all night, but sometimes it shows "4G" for the first few seconds. It is also connected to the charger. Though "Priorisierte Hintergrundaktivität" is not active, there is a check between 2 and 3 almost every day, but not on May 4. The last day before May 4 without a check at night was April 17. This behaviour is not quite deterministic.

Can you confirm that this issue araised only after updating to CWA version 2.1 I have 2.1.2 since May 3, and the issue appeared on May 4 and 6. The exposure log now shows up to four checks in one day. Before CWA version 2.1.2 there had been only one or two: all-exposure-checks06.txt

I don't think a screenshot of COVID-19-Benachrichtigungen will add any useful information. What would you like to see?

dsarkar commented 3 years ago

@kereng5 Thanks for the feedback. Developer asked for the screenshot to see if the slider is greyed out.

vaubaehn commented 3 years ago

@kereng5

The exposure log now shows up to four checks in one day. Before CWA version 2.1.2 there had been only one or two:

This is, because there was/is a bug in OS on how one certain network parameter is determined. Since CWA 2.1.2 it is now worked around that issue, so we Android 6 users get exposure check updates multiple times a day, finally. See: #2881 and #2934.

But looking to your all-exposure-checks06.txt, I see you are/were also affected of #2880, see entries for April 23rd/24th, April 30th/May 1st.

Could you have a look, how much free space there is for your internal storage, please? And provide the screenshot that @dsarkar resp. the devs asked for? Thank you ❤️

kereng5 commented 3 years ago

@vaubaehn There are two entries for April 24 at 02:51 and 12:03 with the same hash and keyCount. Also for May 1, 02:09 and 08:59. When I noticed cases like that last year, I thought the server data would not change in one day. Now I am pleased to see that the four entries for May 3 or 5 all have different data.

Gerätespeicher 7,37 GB von 8,00 GB insgesamt verwendet Android Betriebssystem 3,46 GB Interner Speicher: 3,91 GB von 4,45 GB verwendet

Arbeitsspeicher 833 MB von 898 MB, 93%

The screenshot that @dsarkar resp. the devs asked for: slider_5169

kereng5 commented 3 years ago

Today same as yesterday "Risiko-Ermittlung gestoppt ... Aktualisiert Heute, 02:59". Tap on "Risiko-Ermittlung aktivieren", move the slider to the right. Tap on "x": still "Risiko-Ermittlung gestoppt". Move the slider to the right again, tap on the "back triangle": still "Risiko-Ermittlung gestoppt". Move the slider to the right again, close the CWA and restart it: exposure check is performed with new server data.

dsarkar commented 3 years ago

@kereng5 Thanks for the feedback. Will be forwarded to developers.

kereng5 commented 3 years ago

@MikeMcC399 I have v1.8 (211515000) now. But I think this update happened after the exposure check.

MikeMcC399 commented 3 years ago

@kereng5

I have v1.8 (211515000) now. But I think this update happened after the exposure check.

I assume that this was an automatic update if you are not sure when it happened. It is good that you have the latest version, although unfortunately Google doesn't publish details about the fixes they release. The release-notes only cover the major releases, not the individual releases that mostly come out at monthly intervals.

kereng5 commented 3 years ago

@MikeMcC399

I assume that this was an automatic update if you are not sure when it happened.

When I go to Einstellung > Apps > Google Play-Dienste there is no "Store" and no "App-Details". And in the Google Play Store I cannot find Google Play-Dienste, so I had to wait for the automatic update.

MikeMcC399 commented 3 years ago

@kereng5

When I go to Einstellung > Apps > Google Play-Dienste there is no "Store" and no "App-Details".

Here is what it looks like on my Android 8 device. It's a pity that function isn't available to you on Android 6.

And in the Google Play Store I cannot find Google Play-Dienste, so I had to wait for the automatic update.

The direct link for "Google Play-Dienste" is https://play.google.com/store/apps/details?id=com.google.android.gms and it seems to be hidden from search in Google Play Store. I opened that link in Google Chrome browser on the mobile device and Chrome gave me the option "IN PLAY STORE APP ÖFFNEN". If you try that and it is successful for you, you should just see the option "Deaktivieren" since you already have the latest version of Google Play-Dienste (aka gms). If you didn't have the latest version it would offer to update at that point.

kereng5 commented 3 years ago

The problem did not occur today. Exposure checks at 02:10 and 08:15. The reason could be v1.8 (211515000) or switching the phone off for a minute, which I did yesterday. It is much faster now. Arbeitsspeicher 668 MB von 898 MB, 74% (two days ago: 93%)

Ein-Tim commented 3 years ago

Another user experiencing this: https://twitter.com/biblioman1/status/1390998405187575808?s=21 I asked him for a screenshot of "Einstellungen" -> "COVID-19-Benachrichtigungen". Anything else you need to investigate this (cc @MikeMcC399)

kereng5 commented 3 years ago

"Risiko-Ermittlung gestoppt" has not occured again. I had it only three times on May 4, 6, and 7. Yesterday there was no exposure check at night between 2 and 4, and today it has the timestamp 2:21 in CWA and 2:34 in ENS which means it took 13 minutes (?). Before that 7 minutes was the longest check.

dsarkar commented 3 years ago

@kereng5 Thanks for the feedback.

Can you confirm that with the ENF update "Risiko-Ermittlung gestoppt" did not occur again?
Regarding the duration of the checks, I will consult a developer.

Thanks again. Best, DS

Corona-Warn-App Open Source Team

thomasaugsten commented 3 years ago

On old phone this duration can happen

kereng5 commented 3 years ago

@dsarkar

* Can you confirm that with the ENF update  "Risiko-Ermittlung gestoppt" did not occur again?
CWA 2.1.2 came on May 3. "Risiko-Ermittlung gestoppt" on May 4, 6, and 7. ENF update v1.8 (211515000) on May 7. No "Risiko-Ermittlung gestoppt" on May 8, 9, and 10. I am not quite sure whether "Risiko-Ermittlung gestoppt" on May 7 happened before or after the ENF update, but I guess the update was later.

dsarkar commented 3 years ago

@kereng5 ok, thanks for the information. Let us know if the problem persists or disappeared for good in the next few days, please. Thanks.

Corona-Warn-App Open Source Team

kereng5 commented 3 years ago

"Risiko-Ermittlung gestoppt" happened again today at half past ten. Arbeitsspeicher is at 82%, and the phone is not really slow today. The last exposure check before "Risiko-Ermittlung gestoppt" started at 21:41 (CWA) and finished at 21:45 (ENF). The newest exposure log is attached: all-exposure-checks11.txt

vaubaehn commented 3 years ago

Hi @kereng5 , unfortunately I don't have much time at the moment, but I really want to ask you some more diagnostic questions:

When you (try to) re-enable the slider in CWA, do you get any notification/pop-up/dialog, that contact tracing is now turned on (again)?

For more diagnostics for the next time, could you please try following: First, today go to Android > Settings > Google > Covid-19 Benachrichtigungen and stay on this "start page" of Google's Exposure Notification System (ENS). Close the Screen with the round circle at the buttom of the screen. Then, when you open the next time CWA and you see, that the slider is off in CWA, directly jump to the "start page" of the ENS by clicking on the square at the right in the bottom of your screen, chose the ENS page and open it. Is the slider turned off here, too? Or is it still on, contradictory what CWA displays?

Thanks, V.

Edit: thanks for your information to me from the past days, I'll write something about it later.

kereng5 commented 3 years ago

@vaubaehn

When you (try to) re-enable the slider in CWA, do you get any notification/pop-up/dialog, that contact tracing is now turned on (again)?

I only got a pop-up telling me that I also should activate "Priorisierte Hintergrundaktivität", which I ignore because I open the CWA every day. The pop-up "Covid-19-Benachrichtigungen aktivieren?" did not occur.

I have seen this pop-up just now for the first time, when I tested what happens in CWA and in ENS when I turn the Risiko-Ermittlung off and on again. It behaves differently now than it does in the morning when CWA says "Risiko-Ermittlung gestoppt".

kereng5 commented 3 years ago

Yesterday in the afternoon, when the CWA behaved as it should, the slider in ENS was always following the slider in CWA. Now, in the morning of May 12, the slider in ENS is right in the active position "Covid-19-Benachrichtigungen verwenden", while the slider in CWA is left, inactive, "Risiko-Ermittlung gestoppt". I can move it to the right an click OK on the pop-up "Priorisierte Hintergrundaktivität deaktiviert". The slider is right while the text still says "Risiko-Ermittlung Gestoppt": slider_5170 Sometimes I can see the slider move from right to left, when I return to the CWA, but that is not reliably reproducible. The slider in ENS stays right (on) during all my tests and in Einstellungen-Google it says "Covid-19-Benachrichtigungen an".

I almost never turn the CWA off. It is always in the list of small app screens that I get with the square in the bottom line of my phone. I only turn it off and on again to get out of this "Risiko-Ermittlung Gestoppt" situation. After that CWA says "Risiko-Ermittlung aktiv" and performs an exposure check.

vaubaehn commented 3 years ago

@kereng5 Thank you very much for your detailed description! Imho, they are the most valuable and important to get more insight into what caused the problem!

Obviously the link between CWA and ENS is broken in a certain way, otherwise the sliders would be synchronized, but they're not.

I assume, there could be 2 reasons for this issue, theoretically:

ENS is crashed, and does not respond to API calls from CWA (i.e., does not provide information, whether tracing is enabled). This might have happened in the context of a currently rolling out ENS update.
CWA has a problem. For me this seems to be more likely, as the restarting of CWA sometimes solves the problem.

If CWA has a problem, then there may be either an inconsitency in the persisted data of the tracing status, or for some other reason CWA cannot cleanly determine whether tracing is enabled.

@d4rken Do you think, #3122 may already have an impact on this issue here? As I could see, tracing status will be logged better in CWA 2.3.x - should @kereng5 enable debug log in the upcoming CWA 2.2 though?

@dsarkar @heinezen When you check Google Play Store reviews, you will find a lot of problem reports of this issue for the last days. Could you please verify and consider to forward them to dev department, so that priority of the issue here might be re-evaluated? Thanks a lot! See you, V.

dsarkar commented 3 years ago

@kereng5 @vaubaehn Thanks for the comments here. This issue has been raised internally and developers are aware of the issue.

@kereng5 Could you produce a error log? https://www.coronawarn.app/de/faq/#error_log https://www.coronawarn.app/en/faq/#error_log

Thanks.

kereng5 commented 3 years ago

@dsarkar The FAQ say that I need 2.2 for a error log, and that I should have 2.2.1, but Google Play Store tells me that 2.1.2 is the newest version.

DerVogel2020 commented 3 years ago

@kereng5 It's the same as for the previous updates. In the blog it's said: "Version 2.2 - like previous versions - will be delivered in a controlled rollout and is made available for users in waves. While users can manually trigger an update in Apple’s App Store, this option is not available in the Google Play Store. There, the Corona-Warn-App’s new version can be available up to 48 hours later." https://www.coronawarn.app/en/blog/2021-05-12-corona-warn-app-version-2-2/

kereng5 commented 3 years ago

I am having the "Risiko-Ermittlung Gestoppt" condition again, for the first time in the afternoon.

Next test: I go to Einstellung/Google/Covid-19-Benachrichtigungen. "Covid-19-Benachrichtigungen verwenden" is on, and I move the slider to the left and click the pop-up away. Now the slider is off in ENS and in CWA. I move it to the right in CWA and get both pop-ups: "Covid-19-Benachrichtigungen aktivieren?" (yes, "aktivieren) and "Priorisierte Hintergrundaktivität deaktiviert" (OK). Now the slider in ENS is right again, but CWA still says "Risiko-Ermittlung Gestoppt" as in my screenshot 6 hours ago, where we can see an inconsistency in CWA.

Next test: I turn the phone off for a minute. After restart of the phone the CWA is still in the list of apps that are shown by clicking the square in the bottom line. The small screen in this list still shows "Risiko-Ermittlung Gestoppt". I tap on it and now it says "Risiko-Ermittlung aktiv" and "Aktualisiert: Heute 16:28". That was just some minutes before I saw "Risiko-Ermittlung Gestoppt".

Lu9Bro commented 3 years ago

I have the same problem with my

Device name: Samsung Galaxy S5 mini Android version: 6.0.1 CWA version 2.1.2 and 2.2.1 ENF 18211515000 "Priorisierte Hintergrundaktivität" ON 16 GB total memory 1.13 GB free memory WLAN is available continuously. I switch the device in Offline Mode during the night, and back in Online Mode in the morning.

(I have the 100% same behavior of the CWA app and the slider; I didn’t verify so deeply as @kereng5 did if my ENS behavior is also 100% the same.) The problem exists with CWA version 2.1.2 and 2.2.1.

While the CWA was trying to download the exposure check data and I was watching the device during this process, I got an „URSACHE 9002 Timed out“ with the following details: Screenshot_2021-05-12-07-10-21 crop

Maybe this helps. If you need more tests, let me know.

Ein-Tim commented 3 years ago

One more Twitter user reporting this Problem: https://twitter.com/pantheraleo356/status/1393239658700681216?s=21

kereng5 commented 3 years ago

Meanwhile I have Version 2.2.1, but "Risiko-Ermittlung Gestoppt" did not accur in the last three days.

On May 14 "Anzahl der Schlüssel" or keyCount was 376735 at 03:00, but only 20057 at 07:18, 30358 at 17:28, and 35248 at 21:37, which is less than one-tenth of what it should be. The next day it was back in the normal range with 367569 at 02:43. Only on my phone? Sorry for being off-topic, but this irregularity seems to small to start a new issue.

all-exposure-checks16.txt

MikeMcC399 commented 3 years ago

@kereng5 I don't have any matches for your low numbers on May 14, 2021. They are all 6 figure keyCounts. Your 5 figure ones look suspicious!

timestamp	keyCount
16 May 2021, 15:33	352445
16 May 2021, 11:24	348961
16 May 2021, 07:24	347322
16 May 2021, 03:04	346709
15 May 2021, 22:42	367304
15 May 2021, 18:42	364064
15 May 2021, 12:39	356761
15 May 2021, 08:39	352629
15 May 2021, 04:36	351723
15 May 2021, 00:03	377459
14 May 2021, 19:32	373401
14 May 2021, 15:30	367875
14 May 2021, 11:27	363025
14 May 2021, 07:23	360364
14 May 2021, 03:22	359471

vaubaehn commented 3 years ago

@kereng5 I had a similar issue for the days February 4 and 5. @ndegendogo found out, that it was just a single diagnosis key file that was 14 days old at that time (instead of the full package holding the latest 14 days' diagnosis keys). I'm quite sure that this is not related to the issue here. I think it could either be worth to open a new issue for this, or maybe to extend #2880 - it could be a problem in CWA's caching mechanism of diagnosis key files. @d4rken FYI: https://covidapps.slack.com/archives/C0194ML0MLN/p1612438180080300?thread_ts=1612438180.080300&cid=C0194ML0MLN

Edit: Another reason for this issue could be, that CWA provides the whole diagnosis key package with all files to ENS correctly, but ENS interrupts exposure checking after the first (and probably oldest) diagnosis key file.

Did you already start the error logging like it was suggested above (CWA > App Information > Fehlerberichte)? This could help to identify the reasons for the issue of the original issue with stopped exposure logging slider and the secondary single diagnosis key file, given they will occur again.

ndegendogo commented 3 years ago

@kereng5 I can confirm that your 5-digit numbers are way too low; ~although I couldn't find partial sums of the true set of keys that match your numbers~ => see my newer comments below Incidentally, it looks as if the server had a hickup / slow response times on that day. At 10:42 I had correctly 8 hourly files (since 00:00 UTC) and 14 daily (the previous 14 days). But at 14:47, I had only 9 hourly files instead of 12, plus the 14 daily files as expected. At 18:48 I was back to normal (16 hourly and 14 daily files).

ndegendogo commented 3 years ago

@kereng5 @vaubaehn wow - meanwhile I succeeded to find the sum! @vaubaehn you are on the right track!!!

The hourly numbers for that day are: [0, 36, 169, 481, 207, 309, 695, 691, 966, 1124, 1216, 1270, 1240, 1384, 1406, 1325, 1411, 1039, 1115, 510, 862, 532, 300, ...] (the last numbers I don't have). Your number 20057 at 07:18 is the sum of the 5 hourly files so far (0 + 36 + 169 + 481 + 207 = 893) and the daily file of the previous day (13 May = 19164 keys): 893 + 19164 = 20057

Your number 30358 at 17:28 is the sum of the 15 hourly files so far (0 + 36 + ... + 1406 = 11194) and again the same daily file: 11194 + 19164 = 30358

Your number 35248 at 21:37 is the sum of the 19 hourly files so far (0 + 36 + ... + 1115 = 16084) and again the same daily file: 16084 + 19164 = 35248

ndegendogo commented 3 years ago

And the good news is: although it checked not all files (for whatever reason), at least the newest files were always included in the check.

dsarkar commented 3 years ago

@kereng5 @Lu9Bro If you could provide us with an error reports, once you could reproduce the issue or the issue occurred again, that might help the developers to investigate the issue further. You need CWA version 2.2 for this. See https://www.coronawarn.app/de/faq/#error_log https://www.coronawarn.app/en/faq/#error_log

Thanks.

Corona-Warn-App Open Source Team

vaubaehn commented 3 years ago

@kereng5 and @Lu9Bro : to prevent any misunderstanding regarding @dsarkar 's suggestion - it would mean if you could already turn on the error logging via "CWA > App-Informationen > Fehlerberichte" now, and after the issue occured again, upload the log and provide the ticket numbers here.

Lu9Bro commented 3 years ago

Today, I managed to record the issue using the new "error_log" function. I uploaded the logfile with ID 1C18577E0307D2148D44 .

Unfortunately, I took long time until the error occurred. It occurred at some point in time between 12:59 and 17:27 (MESZ). Tomorrow, I'll try to catch it again and within a shorter time range.

PS: Maybe, you'll see in the logfile an "incompatibilty warning", which is related to #3170 . This is independant of the actual problem, so don't be confused.

dsarkar commented 3 years ago

@Lu9Bro Thank you very much. This has been forwarded to the internal ticketing system. Best wishes, DS

Corona-Warn-App Open Source Team

Lu9Bro commented 3 years ago

Today I've caught the error a bit closer. I uploaded the error log ID D1A61C313C065522CDAC .

And I have some screenshots, which show what is visible to the user. At 12:52 MESZ (=10:52 UTC), everything was still ok: Screenshot_2021-05-20-12-52-10 crop

Somewhere in the meantime until 14:50 MESZ (=12:50 UTC), the problem has occurred. (But I didn’t watch my device in the meantime.) Screenshot_2021-05-20-14-50-59 crop The screenshot says that CWA has updated the risk at 13:13 MESZ (=11:13 UTC). In fact, this timestamp is misleading, because it does not mean “updated successfully”, but just “started to download and process”, which finally has failed.

After I had noticed the failure, I closed the CWA and reopened it. Then everything is “green” again, without any other user action needed. Screenshot_2021-05-20-14-51-49 crop

After this, I uploaded the error log file.

When I look into the log file, the problem seems to occur around 2021-05-20T11:14:48 .

I hope this helps the development team.

dsarkar commented 3 years ago

@Lu9Bro Thank you very much. Will be forwarded!

Corona-Warn-App Open Source Team

cydur commented 3 years ago

Similar problem on my Sony Xperia Z3 Compact, Android Version 6.0.1. CWA Version 2.2.1 Once a day "Risiko Ermittlung" goes off and cannot be reactivated. It is possible to move the switch to on in the "Risiko Ermittlung aktivieren" page but the main page (screenshot) does not change. I have to hard kill the app in the Android Settings -> Apps section. Then after a restart the "Risiko Ermittlung" is back on even without turning the switch.

Screenshot_20210520-194749

kereng5 commented 3 years ago

@dsarkar I have the error log running all the time, but the error doesn't happen any more - ten days in a row now (40 successful exposure checks).

dsarkar commented 3 years ago

@kereng5 OK, thanks for the feedback. Somehow this is good news. If the log file does not grow to big you might want to continue the logging. Or alternatively, stop and delete the current one and start a new error report.

Thanks.

Corona-Warn-App Open Source Team

dsarkar commented 3 years ago

@cydur Thanks for your report. Could you please create an error report, see this comment and the following comments, please: https://github.com/corona-warn-app/cwa-app-android/issues/3093#issuecomment-843133619

Many thanks. Best wishes, DS

Corona-Warn-App Open Source Team

corona-warn-app / cwa-app-android